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,-f(3)  to  identify  current  issues  and  recommendations  for  future  work.  Percep- 
tronics'  own  statement  of  the  issues  related  to  aircrew  emergency  decision 
training  forms  the  initial  paper  in  this  report.  The  bulk  of  the  report  is 
made  up  of  condensations  of  the  individual  papers  and  presentations  from  the 
conference A  section  entitled  "Epilogue  -  30  minutes  Over  Florida"  describes 
an  aircraft  emergency  experienced  by  one  of  the  participants  of  the  conference, 
just  ten  days  after  the  conference  ended.  The  conference  agenda  and  a  list  of 
participants  are  included  as  appendices.  Emergency  training  programs  were  dis¬ 
cussed  from  a  wide  variety  of  standpoints,  including  media  and  methods,  sequent 
ing  of  instruction,  the  role  of  the  instructor,  validation  and  evaluation  of 
training  programs,  problems  in  field  implementation,  and  the  variety  of  aud¬ 
iences  which  emergency  decision  training  programs  are  required  to  address.  The 
role  of  emergency  scenarios  in  training  program  development  was  reviewed  and 
the  importance  of  validating  the  procedural  doctrine  to  be  applied  in  response 
to  scenarios  was  stressed.  The  need  for  developing  scenarios  which  use  realis¬ 
tic  cues,  information  rates  and  time  frames  was  pointed  out.  Both  prospective 
and  retrospective  approaches  to  scenario  generation  are  necessary  to  ensure  tha 
a  compreshensi ve  and  relevant  set  of  training  problems  are  developed. 

Some  of  the  more  frequent  recommendations  and  conclusions  which  were  brought 
forth  at  the  conference  include  the  following:  (1)  performance  requirements 
in  aircraft  emergency  situations  range  from  rote  responding  to  complex  analysis; 

(2)  emergency  decision  training  should  address  this  range  of  requirements; 

(3)  training  at  all  levels  of  aircrew  proficiency  should  be  considered,  not 
just  at  initial  levels;  (4)  decision  theory  concepts  can  be  taught  to  aircrews; 
however,  decision  theory  must  be  linked  to  practical  applications  to  gain 
acceptance  and  use;  (5)  areas  of  importance  include:  option  generation,  estab¬ 
lishing  utilities,  personal  decision  rules,  and  preplanning/rehearsal;  (6)  In¬ 
structional  System  Development  (ISD)  personnel  should  ensure  that  the  systems 
knowledge  necessary  for  emergency  decisions  is  not  omitted  from  training  for 
specific  aircraft  systems;  (7)  training  should  be  carried  out  in  a  manner  that 
resembles  the  real  life  situation  (e.g.,  via  scenarios)  in  order  to  facilitate 
transfer;  (8)  ancillary  cues  should  be  defined  and  included  in  training  scenarii 
simulations;  (9)  design/development  data  and  field  performance  data  (incidents, 
accidents)  should  be  fed  to  ISD  personnel  to  update  training  regularly; 

(10)  special  attention  should  be  paid  to  teaching  difficult  component  skills 
individually  and  to  developing  strategies  to  deal  with  persistent  performance 
problems. 
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This  report  is  the  record  of  a  two  and  on-half  day  working  conference 
held  in  San  Francisco,  California,  on  November  28,  29  and  30,  1978.  The 
conference  brought  together  selected  members  of  the  military  research 
support  community,  military  contractors,  instructor  pilots,  and  other 
individuals  concerned  with  aircrew  training,  safety  research,  and  behav¬ 
ioral  decision  theory.  Purposes  of  the  conference  were:  (1)  to  review 
the  state  of  the  art  of  aircrew  emergency  decision  training;  (2)  to 
review  the  implications  of  behavioral  decision  theory,  safety  research 
and  training  technology  for  aircrew  emergency  training;  and  (3)  to  iden¬ 
tify  current  issues  and  recommendations  for  future  work.  The  conference 
was  organized  by  Perceptronics  in  connection  with  a  sponsored  research 
project  entitled  Instructional  Systems  Development  and  Situational 
Emergency  Training,  funded  through  contract  F49620-78-C-0067  with  the 
Air  Force  Office  of  Scientific  Research. 

Perceptronics'  own  statement  of  the  issues  related  to  aircrew  emergency 
decision  training  forms  the  initial  paper  in  this  report.  The  bulk  of 
the  report  is  made  up  of  condensations  of  the  individual  papers  and 
presentations  from  the  conference.  The  section  entitled  "Epilogue  - 
30  Minutes  Over  Florida"  describes  an  aircraft  emergency  experienced  by 
Lieutenant  Mike  Bryant,  one  of  the  participants  of  the  conference,  just 
ten  days  after  the  conference  ended.  The  conference  agenda  and  a  list 
of  participants  are  included  as  appendices  to  this  report. 

It  was  generally  agreed  that  the  conference  was  successful  in  reaching 
its  goals.  This  was  due  in  no  small  part  to  the  speakers  and  other 
participants  and  their  efforts  must  be  acknowledged.  The  contributions 
of  the  instructor  pilots  and  other  flying  personnel  in  attendance  deserve 
special  recognition.  While  it  is  hoped  that  this  report  captures  the 
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essence  of  the  technical  presentations  and  discussions  at  the  conference, 
Perceptronics  remains  solely  responsible  for  the  accuracy  of  the  material 
contained  herein. 
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DECISION  TRAINING  AND  THE  AIRCRAFT  EMERGENCY  PROBLEM 

Luigi  Lueaocini 
Pereeptvonios,  Inc. 

An  emergency  is  commonly  defined  as  an  unexpected  occurrence  of  a  set  of 
circumstances  which  calls  for  immediate  judgment  and  action  to  avoid 
undesirable  consequences.  The  standard  emergency  response  expected  of 
every  aircrew  is  three-fold:  (a)  to  maintain  aircraft  control,  (b)  to 
analyze  the  situation  and  take  proper  action,  and  (c)  to  land  as  soon  as 
practicable.  In  the  broader  context  of  flying  safety,  however,  aircrews 
are  expected  to  do  more  than  skillfully  resolve  immediate  full-blown 
emergencies.  It  is  equally  important  that  they  actively  avoid  situations 
which  can  lead  to  emergencies  and  that  they  recognize  the  early  signs  of 
an  impending  emergency  and  take  corrective  action  before  the  situation 
assumes  crisis  proportions. 

Emergency  preparedness,  under  this  view,  goes  beyond  the  capability  to  make 
very  rapid,  accurate  decisions  under  intense  time  pressure.  It  is  a  truism 
among  experienced  flying  personnel  that  there  is  usually  more  than  enough 
time  to  deal  with  most  emergencies  and,  further,  that  it  is  not  the  first 
mishap,  or  even  the  second,  that  kills  pilots,  but  the  third.  An  aircraft 
emergency  might  be  viewed,  then,  as  a  sequence  of  events  (and  decisions), 
which,  if  not  recognized  and  resolved  at  an  earlier  stage,  culminate  in  a 
crisis.  If  so,  emergency  preparedness  training,  and  more  specifically, 
emergency  decision  training,  must  accommodate  the  broader  range  of  situa¬ 
tions  and  skills  that  this  conception  embraces. 

In  the  strict  sense,  decision  making  can  be  viewed  as  the  efficient 
translation  of  high  quality  information  into  appropriate  action  by  a 
rational  decision  maker  using  effective  decision  strategies.  While  this 
may  be  an  adequate  description  of  decision  analysis,  a  slightly  broader 
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view  of  decision  making  is  necessary  to  encompass  the  decision  activities 
important  in  the  practical  setting  of  aircrew  emergency  decision  training. 

As  Nickerson  and  Feehrer  (1975)  point  out,  decision  making  involves  a 
number  of  overlapping  aspects  or  phases  of  activity,  which  might  best  be 
conceived  of  as  a  series  of  related  problem  solving  tasks.  In  our  view, 
the  following  breakdown  is  useful  in  characterizing  the  general  area  of 
decision  making: 

(1)  Diagnosis. 

(a)  Problem  recognition. 

(B)  Information  acquisition  and  evaluation. 

(2)  Decision-Making. 

(a)  Problem  structuring  and  development  of  alternatives. 

(b)  Evaluation  of  alternatives  and  selection  of  a  course  of 
action . 

(3)  Decision  Execution. 

(a)  Implementation  of  action  alternative. 

(b)  Monitoring  of  implementation  and  evaluation  of  results. 

The  training  of  emergency  decision  makers  should  accommodate  the  wide 
range  of  functions  and  underlying  skills  that  a  breakdown  such  as  this 
implies.  A  second  obvious  consideration  for  those  developing  and  providing 
emergency  decision  training  is  incorporation  of  the  sys tern- spec i fic  elements 
of  knowledge  appropriate  to  the  particular  aircraft  system  involved.  A 
third  general  concern  is  with  the  economics  of  the  training  situation,  namely 
resources,  resource  policies,  and  other  factors  which  shape  and  constrain 
the  instructional  approaches  that  might  be  utilized.  A  fourth  consideration 
is  the  degree  to  which  training  should  be  specific  or  general  in  nature. 
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The  question  of  general  versus  specific  approaches  to  decision  training 
is  not  a  new  one  and  has  been  raised  repeatedly  by  those  who  have  sur¬ 
veyed  the  field  (Goodman,  Fischhoff,  Lichtenstein  and  Slovic,  1976; 
Kanarick,  1969;  Nickerson  and  Feehrer,  1975).  All  are  in  agreement  that 
our  understanding  of  this  issue  is  hampered  by  a  lack  of  theoretical 
guidance  which  is  anchored  to  a  solid  body  of  experimental  evidence. 

Current  research  in  cognitive  science  holds  some  promise  for  those  who 
hope  to  identify  the  common  elements  in  decision-making  strategies  used 
in  different  tasks  as  a  basis  for  developing  general  approaches  to  decision 
training. 

Work  at  Perceptronics ,  Inc.  (Saleh,  Leal,  Lucaccini,  Gardiner  and  Hopf- 
Weichel,  1978)  has  led  to  the  development  of  an  approach  to  the  systematic 
identification  and  classification  of  task-specific  decision  functions  and 
related  training  objectives.  This  methodology  has  relevance  to  the  analy¬ 
sis  and  improvement  of  aircrew  emergency  training  programs,  particularly 
for  the  structuring  of  scenarios  and  training  exercises.  It  is  our  hope 
that  this  methodology  will  also  facilitate  the  development  and  testing  of 
approaches  to  decision  training  which  permit  some  degree  of  generality. 

The  convening  of  a  conference  to  review  aircrew  emergency  decision  training 
is  an  ambitious  attempt  to  deal  with  a  difficult  and  important  problem. 

The  topic  itself  is  a  complex  one  which  represents  the  overlap  of  several 
disparate  disciplines  and  interests.  Moreover,  the  growing  pressure  on 
training  systems  to  continue  to  produce  highly  skilled  flying  personnel 
in  the  face  of  increasingly  complex  aircraft  systems  and  decreasing 
resources  for  training  serves  to  intensify  the  problem.  Two  outcomes  of 
this  conference  will  be  of  value.  The  first  is  to  call  attention  to  this 
important  area.  The  second  is  to  provide  stimulation  and  guidance  to  those 
wishing  to  advance  the  current  state  of  the  art.  The  papers  summarized  in 
this  report  reflect  the  efforts  of  the  participants  towards  these  goals. 
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STATEMENT  OF  WORKSHOP  GOALS 

Major  Jack  Thorpe 

Air  Force  Office  of  Scientific  Research 

One  area  of  interest  in  the  flight  and  technical  training  program  of 
AFOSR  is  how  critical  decisions  are  made  by  aircrew  members.  This  is 
related  to  an  understanding  of  the  skills  which  underlie  successful 
decision  making  and  how  such  skills  can  be  developed  through  training 
programs. 


Aircrew  members,  both  pilots  and  others  on  the  flying  team,  can  be 
characterized  in  some  important  ways.  They  are  highly  screened,  care¬ 
fully  selected  individuals.  Extensive  and  expensive  training  programs 
are  used  to  bring  their  flying  skills  to  a  level  of  high  quality.  The 
responsibility  they  bear  when  carrying  out  missions  is  enormous  in  terms 
of  life,  equipment,  and  mission  achievement  and  is  rarely  paralleled  in 
other  professions. 

Aircrews  are  called  on  to  perform  in  an  environment  for  which  man  is 
imperfectly  adapted.  Unusual  visual  demands,  the  dynamics  of  flight, 
and  an  accelerated  time  frame  for  action  typify  flight  in  high  perform¬ 
ance  aircraft.  These  are  among  the  factors  which  complicate  the  human 
performance  setting,  and  make  decisions  more  critical  and  the  outcome 
of  such  decisions  potentially  more  catastrophic  than  in  many  other 
situations.  The  varied  and  complicated  mission  profiles  and  maneuvers 
of  military  aircrews  in  comparison  with  commercial  crews  require  a  very 
complex  skill  repertoire.  The  training  time  available  to  build  this 
repertoire,  on  the  other  hand,  is  of  necessity  limited.  We  are  faced, 
therefore,  with  a  very  challenging  and  demanding  training  problem. 

Two  approaches  to  aircrew  emergency  decision  training  that  will  be 
discussed  at  this  meeting  should  be  defined.  The  first,  Boldface, 
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is  a  standard  Air  Force  approach  which  has  been  in  existence  for  many 
years.  In  Boldface,  the  pilot  is  required  to  memorize  that  part  of  his 
flight  manual  which  describes  the  response  procedures  for  various  emergency 
conditions,  those  procedures  being  printed  in  Boldface  type.  Pilots  are 
tested  regularly  on  their  retention  of  these  critical  actions  by  frequent 
paper  and  pencil  tests  and  by  less  frequent  simulator  spot  checks.  The 
emphasis  in  a  Boldface  training  approach  is  on  memorizing  these  actions 
for  broad  classes  of  emergencies. 

The  second  approach.  Situational  Emergency  Training  (SET),  attempts  to 
deal  with  the  special  tailoring  of  responses  needed  when  the  standard 
response  under  Boldface  may  not  be  sufficient.  That  is,  a  SET  approach 
recognizes  that  each  emergency  situation  has  unique  properties,  and, 
while  a  sequence  of  Boldface  activies  might  be  appropriate  for  a  high 
percentage  of  a  class  of  situations,  occasionally,  the  good  pilot  will 
have  to  modify  a  Boldface  response. 

In  just  about  every  aircrew  training  program,  these  two  approaches  are 
mixed  to  some  extent;  but  with  one  aircraft,  the  operational  community 
has  made  a  deliberate  step  towards  developing  a  situational  approach. 

This  is  the  F-15  air  superiority  fighter  training  program,  and  its  use 
and  development  of  SET  will  be  described  by  following  speakers  in  more 
detail.  Other  speakers  will  address  simulator  technology  and  innovations 
in  ground  training  programs,  especially  ground  training  of  emergency 
decision  making. 

This  workshop  will  be  successful  in  helping  AFOSR  set  priorities  for 
future  research  support  if  it  extends  our  knowledge  and  insight  regarding 
how  critical  decisions  are  made  and  how  the  current  state-of-the-art  of  be¬ 
havioral  decision  theory  and  training  technology  can  be  utilized  to  improve 
aircrew  emergency  decision  skills.  Another  important  goal  for  this  workshop 
is  to  stimulate  a  more  regular  and  comprehensive  dialog  among  the  skilled 
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aviators,  the  members  of  the  accident  investigation/safety  research 
community,  the  decision  theorists,  and  the  training  technologists  in 
attendance. 
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ONR  PROGRAMS  AND  EMERGENCY  TRAINING 


Henry  Halff 

Office  of  Naval  Research 

The  Personnel  and  Training  Research  programs  at  ONR  support  two  kinds  of 
research  which  might  be  relevant  to  the  concerns  of  this  workshop.  The 
first  type  of  research  is  concerned  with  the  psychological  bases  for 
generative,  knowledge-based,  computer-assisted  instruction.  The  second 
area  is  concerned  with  the  cognitive  processes  and  structures  underlying 
skilled  problem  solving  and  information  processing  in  real  world  tasks. 

Jack  Thorpe  and  I,  in  discussing  this  workshop,  were  able  to  think  of 
four  ways  in  which  these  research  areas  might  apply  to  the  problems  which 
arise  in  handling  emergencies. 

First,  Jack  mentioned  that  one  cause  of  accidents  is  pilots'  failure  to 
deal  with  the  ongoing  task  of  flying  the  airplane  while  he  or  she  is  dealing 
with  the  emergency  itself.  This  phenomenon  strikes  me  as  being  analogous 
to  that  found  in  many  memory  experiments  involving  changes  in  context  between 
learning  and  recall.  The  effect  of  such  changes  is  usually  to  make  certain 
well-learned  responses  unavailable,  and  a  similar  effect  might  be  obtained 
if  current  training  practices  led  pilots  to  change  psychological  contexts  in 
dealing  with  emergencies.  Certainly  situational  training,  to  the  extent 
that  it  treats  emergencies  in  the  context  of  normal  flying  operations,  might 
be  better  in  this  regard  than  Boldface  procedures. 

A  second  problem  mentioned  by  Jack  was  the  tendency  of  pilots  to  cling  to 
one  hypothesis  concerning  the  nature  of  a  malfunction,  long  after  the  readily 
available  indicators  would  clearly  disconfirm  the  hypothesis.  In  Bayesian 
terms,  this  behavior  reflects  a  failure  to  buy  information  in  situations 
where  such  a  purchase  would  have  a  very  low  cost-benefit  ratio.  But  a 
cognitive  scientist  might  interpret  such  behavior  in  terms  of  scripts,  frame¬ 
works  by  which  people  comprehend  particular  situations,  limit  the  range  of 


potential  actions,  and  determine  which  features  of  the  situation  are 
relevant  to  those  actions.  It  is  not  surprising,  therefore,  that  an  inappro¬ 
priate  script  could,  by  idiosyncratic  restriction  of  attention,  deny  the 
pilot  access  to  those  features  of  the  situation  which  would  cause  him  to 
exchange  the  script  for  one  more  appropriate.  While  such  problems  may  not 
be  entirely  avoidable,  it  should  be  possible  to  discover  by  empirical  means, 
and  control  through  training, those  aspects  of  situations  which  determine 
script  selection. 

A  third  interesting  feature  of  emergency-handi ing  behavior  which  Jack 
mentioned  to  me  was  the  failure,  particularly  of  novices,  to  consider  the 
consequences  of  their  actions.  Horror  stories  of  pilots  ejecting  in  bizarre 
circumstances  and  the  reported  practice  of  pilots  flying  ahead  of  their  planes 
indicate  that  mental  models  of  the  aircraft  and  its  systems  play  a  key  role 
in  diagnosing  malfunctions  and  dealing  with  emergencies.  Research  on  the 
mental  modeling  of  real-world  systems  is  new  in  cognitive  psychology  and  not 
without  controversy.  The  topic,  however,  is  central  to  other  important 
issues  such  as  computer  reasoning,  mental  imagery,  and  the  relationship  of 
procedural  to  declarative  knowledge.  It  therefore  provides  a  dramatic 
example  of  how  research  on  a  relatively  narrow  applied  problem  such  as 
emergency  training  can  illuminate  a  host  of  basic  research  issues. 

A  fourth,  and  last,  issue  that  Jack  and  I  discussed  was  the  possibility  of 
partially  or  fully  automating  situational  emergency  training.  We  have,  in 
other  fields,  achieved  a  certain  measure  of  success  in  using  a  computer- 
resident,  rule-based  system  to  represent  an  expert's  procedural  skills.  Such 
systems  could  be  used  to  evaluate  a  student's  knowledge  of  the  skills  and  to 
generate  cases  on  an  individual  basis  which  are  appropriate  to  the  trainee's 
pattern  of  strengths  and  weaknesses.  Such  systems  could  not,  I  fear, 
represent  the  modeling  capacity,  discussed  above,  beyond  a  very  superficial 
level,  but  they  could  serve  as  a  useful  tool  to  extend  instructors'  resources, 
relieve  them  of  some  of  the  more  routine  chores,  and  increase  the  pace  of 
situational  training. 


SHOULD  PILOTS  NEED  TO  MAKE  EMERGENCY  DECISIONS? 

Ward  Edwards 

Social  Science  Research  Institute , 

University  of  Southern,  California 

In  1958,  Alex  Williams,  a  distinguished  human  factors  specialist,  wrote  a 
long-forgotten  technical  report  on  tactical  decision  making  by  fighter 
pilots.  At  the  time,  I  was  relatively  young,  not  yet  a  pilot,  and  had  the 
familiar  layman's  awe-struck  view  of  the  fighter  pilot  as  Jimmy  Stewart, 
bashful  but  God-like  in  crisis.  So  Alex  Williams'  thesis,  though  completely 
convincing  to  me,  was  also  quite  startling.  It  has  remained  in  my  head  to 
this  day,  and  strongly  colors  all  of  my  thinking  about  what  one  should  do 
in  preparing  for  potential  emergencies. 

That  thesis  was  very  simple,  and  very  persuasive.  It  was  simply  that 
fighter  pilots  do  not  in  fact  make  tactical  decisions.  If  properly  trained, 
they  simply  recognize  pre-specified  situations,  whether  of  a  routine  or  an 
emergency  nature,  and  respond  to  them  as  they  have  been  carefully  trained 
to  respond. 

This  philosophy  came  to  its  fullest  fruition  later,  in  the  manned  space 
flight  program.  I  was  a  close  observer  of  the  processes  by  which  the 
astronauts  were  trained  to  meet  emergencies  in  space,  and  in  fact  I  helped 
in  the  creation  of  some  of  the  elaborate  scenarios  used  in  the  training 
process.  As  most  of  you  know,  the  entire  Mission  Control  facility  in 
Houston  was  used  as  a  gigantic  form  of  training  device,  both  for  ground 
controllers  and  for  the  astronauts  themselves.  A  small  staff  under  the 
direction  of  one  man  was  responsible  for  inventing  scenarios,  keyed  in 
detail  to  specific  missions,  in  which  controllers  and  astronauts  alike 
had  to  deal  with  emergency  situations. 

In  this  context,  I  learned  more  about  emergencies.  Of  course, it  was  one 
of  NASA's  major  goals  never  to  lose  an  astronaut  during  a  mission.  To 
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accomplish  this,  they  not  only  built  extraordinary  layers  of  redundancy 
into  their  systems,  but  also  developed  extremely  elaborate  contingency 
plans  for  what  to  do  in  case  of  any  reasonable  set  of  equipment  or  com¬ 
ponent  failures  that  might  occur.  These  plans,  called  mission  rules, 
did  in  fact  provide  so  carefully  for  so  many  possible  kinds  of  failures 
that  I  had  an  extremely  hard  time,  working  in  collaboration  with  the 
simulation  staff,  in  constructing  any  situation  to  which  anyone,  be  he 
astronaut  or  controller,  had  to  make  a  decision.  And  the  head  of  the 
simulation  staff,  Harold  Miller,  commented  to  me  late  in  the  program  that 
no  emergency  ever  in  fact  arose  in  a  real  flight  that  had  not  previously 
been  included  in  one  of  the  simulaton  runs.  (For  a  report  of  my  work, 
see:  Controller  Decisions  in  Space  Flight.  In:  Applications  of  Research 
on  Human  Decision  Making ,  1968.  Washington,  D.C.  NASA  Scientific  & 

Technical  Information  Division,  1970,  93-106.) 

Preplanning,  then,  turns  the  problem  of  decision  making  into  the  much 
simpler  and  more  tractable  problem  of  situation  recognition.  Once  a 
situation  has  been  recognized,  there  should  be,  and  can  be,  a  pre-specified 
and  known  order,  to  deal  with  it.  The  only  "decision"  a  pilot,  or  anyone 
else,  must  then  make  is  simply  to  figure  out  which  of  the  numerous  possible 
pre-specified  situations  he  in  fact  is  in. 

How  idealistic  is  this?  Not  very,  I  think.  The  basic  tool,  instinctively 
used  by  virtually  any  teacher  and  systematically  used  in  NASA-like  contexts, 
is  the  scenario.  A  scenario  is  simply  a  preplanned  set  of  situations  for 

the  trainee  to  recognize,  so  that  he  can  practice  both  the  recognition  and 

the  pre-specified  response  to  the  situation.  The  standard  engine-failure 
drill  given  to  every  student  pilot  is  an  example--though  a  crude  one. 

The  goal,  I  think,  should  be  exactly  the  one  Harold  Miller  enunciated. 

Never  let  the  pilot  encounter  a  situation  that  he  hasn't  practiced  before. 
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What  can  go  wrong?  Lots  of  things. 

(1)  Too  often,  simulators  are  not  used  for  this  purpose.  Instead 
of  using  a  systematic  scenario,  carrying  through  some  plausible 
set  of  malfunctions  or  other  emergency  conditions  to  their 
natural  conclusion,  teachers,  especially  when  using  simulators, 
sometimes  pile  problem  on  top  of  problem  until  all  resemblance 
of  realism  is  lost--along  with  the  patience  and  tolerance  of 
the  trainee.  I  think  we  should  spend  more  time  than  is  now 
common  in  pilot  training  (at  least  as  I  understand  it)  in 
developing  "emergency"  scenarios  for  simulators  in  such  a  way 
that  they  make  sense  and  are  believable.  The  appropriate 
responses  to  those  scenarios  should  be  carefully  thought 
through,  and  the  trainee  should  clearly  understand  both  what 
they  are  and  why  they  are  appropriate.  I  return  to  the  thinking- 
through  process  below. 

(2)  The  set  of  possible  emergencies  cannot  be  exhausted  in  any  kind 
of  training  process.  Consequently,  pilots  will  inevitably  en¬ 
counter  situations  they  have  not  met  before,  have  not  thought 
about  before,  and  must  therefore  respond  to  on  the  spur  of  the 
moment. 

There  is  truth  to  this  argument,  but  less  of  it  than  meets  the 
eye.  Weather  is,  of  course,  infinitely  variable.  Yet  we  find 
it  relatively  straightforward  to  use  a  quite  limited  number  of 
categories  to  describe  the  aspects  of  it  most  relevant  to  the 
pilot.  Ceilings,  visibilities,  icing,  thunderstorms,  rain, 
snow--these  are  rather  simple  ideas,  yet  most  of  what  a  pilot 
needs  to  know  about  weather  emergencies  can  be  pre-specified 
by  use  of  them.  And  the  set  of  inside-the-plane  emergencies 
is  far  sparser--at  least  in  general  aviation. 


(3)  We  cannot,  and  should  not,  expect  any  training  procedure  to 

result  in  perfect  safety.  For  one  thing,  selection  and  training¬ 
time  requirements  would  be  too  high.  For  another,  even  highly 
trained  and  highly  competent  pilots  are  vulnerable  to  system 
errors.  From  wh at  I  have  read  in  the  newspapers,  there  is 
literally  nothing  that  pilots  of  the  PSA  jet  that  went  down 
in  San  Diego  could  have  done;  the  fundamental  problem  was  that 
light-plane  touch-and-goes  at  Lindbergh  were  being  controlled 
from  Miramar.  That  seems  to  me  to  be  a  clear  case  of  system 
error. 


(4)  Finally,  the  redefinition  of  a  pilot's  role  from  decision  maker 
to  situation  recognizer  is  unlikely  to  be  popular  with  pilots. 

Even  as  we  surrender  more  and  more  control  of  plane  and  mission 
to  systems  and  procedures,  and  to  the  black  boxes  and  remote  people 
necessary  to  implement  them,  and  notice  that  safety  improves  as 
a  result,  we  resent  it.  I  always  fly  IFR  in  VFR  weather.  And 
I  always  resent  the  delays,  inconveniences  of  routing  and  the 
like  that  result. 

Now  I  want  to  turn  to  two  other  questions.  One  is,  how  should  preplanned 
responses  to  emergencies  be  chosen?  Fortunately,  this  is  relatively  simple. 

An  airplane  is  not  a  particularly  complicated  piece  of  equipment,  compared, 
for  example,  to  a  space  vehicle  plus  its  ground  control.  The  set  of  options 
available  in  a  particular  situation  is  generally  sparse.  Often,  there  is 
only  one.  If  my  engine  quits,  and  I  am  unable  to  restart  it,  I  have  no 
option  other  than  to  look  for  a  field.  In  the  presence  of  a  sparse  set  j  .j 

of  options,  it  may  not  require  much  sophisticated  analysis  to  pick  the  } 

best  one.  If  it  does,  sophisticated  methods  are  indeed  available.  In  j 

fact,  my  own  field,  called  decision  analysis,  provides  them.  The  funda-  I 

mental  principle  of  decision  analysis  is  that  every  decision  should,  and 

does,  depend  on  the  answers  to  two  questions:  what  is  at  stake,  and  what  ] 
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are  the  odds.  In  airplane  emergencies,  the  stakes  are  often  quite  well- 
defined:  people  and  plane.  The  odds  are  harder  to  get  at.  But  a  highly 
sophisticated  technology  for  obtaining  numerical  assessments  of  odds  exists, 
and  is  now  in  use  for  much  more  difficult  and  important  decisions  than  the 
ones  we  are  considering. 

Systematic  use  of  assessments  of  stakes  and  odds  has  been  rare  in  aviation 
contexts.  But  it  is  perfectly  feasible.  In  cases  in  which  sophisticated 
analysis  is  relevant  at  all,  which  are  probably  rare,  that,  in  my  view, 
is  the  kind  that  should  be  used. 

The  other  question,  is  in  a  sense,  more  interesting.  It  is  this:  if  a 
pilot  is,  or  should  be,  basically  a  situation  recognizer  (plus  a  book  of 
preplanned  responses  and,  in  some  cases,  a  servo-mechanism),  how  is  he  to 
recognize  situations? 

Inside  the  cockpit,  most  situation  recognition  is  done  by  what  might  be 
called  template  matching.  That  is,  a  specific  set  of  stimuli  define  a 
situation.  Such  a  set  of  stimuli  is  like  a  template.  If  the  situation 
you  are  in  corresponds  to  that  template,  you  make  the  response  appropriate 
to  it.  Obviously,  template  matching  can  be  done  only  if  we  have  a  complete 
set  of  templates.  And  equally  obviously,  given  human  fallibility,  our  set 
of  templates  can  never  be  quite  complete.  However,  if  only  a  few  low- 
probability  templates  are  omitted,  the  situations  in  which  the  pilot  must 
think  for  himself  on  the  spur  of  the  moment  can  be  reduced  to  a  minimum. 

And  that,  in  my  view,  is  a  very  good  idea.  As  a  pilot,  I  consider  my  own 
ability  to  think  clearly  on  the  spur  of  the  moment  to  be  reasonably 
marginal . 

Outside  the  cockpit,  the  situation  is  different.  Flying  into  a  deteriorating 
weather  situation  in  a  light  plane  is  a  nice  example.  As  turbulence  in¬ 
creases,  ice  accumulates,  and  the  weather  reports  from  ahead  of  you  become 
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increasingly  ominous,  when  do  you  decide  to  turn  and  go  back? 

At  least  to  some  extent,  this  too  can  be  a  template  matching  situation. 

The  concept  of  "personal  minimums"  illustrates  what  I  mean.  Though  it  is 
legal,  I  have  never  flown  an  approach  to  a  200-ft.  ceiling;  400  is  my 
personal  minimum. 

In  some  situations,  template  matching  must  be  replaced  or  supplemented  by 
probabilistic  inference.  We  usually  do  this  informally,  but  there  are 
very  useful,  very  formal  ways  of  doing  it  also.  Unfortunately,  the  formal 
procedures  are  not  well -adapted  to  the  cockpit.  But  they  are  very  well 
adapted  indeed  to  preplanning.  In  fact,  they  are  the  techniques  one  uses 
to  answer  the  question:  what  are  the  odds? 

So,  let  me  summarize.  In  at  least  90%  of  the  instances  that  I,  myself,  know 
of,  the  necessity  for  pilot  decision  making  in  the  air  in  an  emergency 
situation  was  the  result  of  failure  of  prior  analysis  on  the  ground.  In  my 
view,  if  the  problem  is  created  on  the  ground,  its  solution  should  be  found 
there  too.  After  it  has  been,  what  remains  to  be  done  is  to  teach  that 
solution  to  the  pilots  themselves. 
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COMMENT 


Paul  Slovic 

Decision  Research  Branch ,  Perceptronics 

There  is  some  interesting  evidence  from  other  fields  which  supports  Ward 
Edwards'  remarks  about  decision  making  in  aircraft  emergency  situations. 
Studies  of  chess  masters  by  DeGroot  and  by  Chase  and  Simon  seem  to  indicate 
that,  more  than  anything  else,  chess  masters  are  excellent  situation 
recognizers.  Similarly,  studies  by  Elstein  and  Shulman  of  skilled 
medical  diagnosticians  show  that  diagnostic  capability  of  physicians  is 
surprisingly  situation-specific.  In  both  cases,  practice  that  involves 
exposure  to  a  wide  range  of  specific  cases  or  problems  appears  to  underlie 
skilled  performance.  Extraordinary  powers  of  reasoning  and  forethought 
do  not  seem  to  be  associated  with  mastery  level  performance.  Experience 
in  a  wide  range  of  situations,  preplanning  of  responses,  and  development  of 
routinized  programs  for  processing  information  seems  most  appropriate. 

Research  on  judgment  and  decision  making  shows  the  importance  of  preplanning 
and,  wherever  possible,  the  automation  of  decision  making.  Findings  from 
research  on  the  diagnostic  performance  of  physicians  that  support  this 
view  include  the  following: 

(1)  Frequent  failure  to  generate  a  relevant  hypothesis  set. 

(2)  A  tendency  to  overemphasize  positive  (confirming)  evidence  and 
to  underemphasize  disconfirming  evidence. 

(3)  Difficulties  in  integrating  several  pieces  of  information  into 
implications  about  a  hypothesis. 

(4)  A  tendency  to  ignore  base  rate  considerations. 

(5)  Interpretation  of  data  to  fit  preconceived  hypotheses  only. 

How  does  one  develop  adequate  preplanning  to  avoid  such  problems?  The  pro¬ 
cess  of  combining  information  to  reach  a  decision  may  be  easier  to  model  than 
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we  think.  Very  specific  and  simple  models  of  expert  decision  behavior  can 
be  developed,  based  on  detailed  observations  made  over  a  period  of  time. 

In  pilot  training,  the  behavior  of  experts  could  be  captured  through  such 
modeling  techniques  taught  to  trainees. 

Another  important  aspect  of  preplanning  is  the  ability  of  the  planner  to 
imagine  in  advance  how  various  contingencies  will  appear  to  the  trainee, 
should  they  come  about.  Two  problems  emerge  here.  For  one  thing, 
information  may  be  responded  to  differently  in  the  real-life  situation 
than  in  practice.  For  another,  our  true  preferences  may  not  surface  in 
the  practice  situation.  These  effects  are  similar  to  the  context  effects 
mentioned  earlier  by  Henry  Halff.  The  analyst  involved  in  preplanning 
needs  to  be  aware  of  such  effects  and  of  the  issues  involved  in  determining 
whether  preplanned  preferences  and  responses  are  better  than  on-the-spot 
reactions  as  guides  to  action. 
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RESOURCE  MANAGEMENT  IN  PRESENT  AND  FUTURE  AIRCRAFT  OPERATIONS 

John  Lauber  and  Renwiok  Curvy 
NASA-Ames  Research  Center 

A  human  factors  research  group  at  NASA-Ames  is  studying  commercial 
aircrew  behavior,  particularly  from  the  standpoint  of  the  captain  of  the 
crew.  The  work  focusses  on  resource  management,  which  is  currently  defined 
as:  "The  application  of  specialized  interpersonal  and  information  pro¬ 
cessing  skills  to  achieve  a  crew  structure  and  process  that  effectively 
and  efficiently  utilize  available  human  and  material  resources  in  attaining 
system  objectives."  The  resource  management  problem  deals  with  the  question 
of  how  well  crew  members,  particularly  the  captain,  use  the  resources 
available,  namely,  information,  hardware,  other  people,  and  other  resources 
(e.g.,  ATC). 

As  part  of  the  overall  research  program,  a  study  was  carried  out  using 
twenty  airline  crews,  each  run  on  a  747  airline  training  simulator,  to 
observe  resource  management  problems  during  full  mission  simulations.  Each 
crew  flew  a  mission  or  line  trip  which  originated  at  Dulles,  included  a 
30-minute  stop-over  at  Kennedy,  and  proceeded  on  to  London.  The  Dulles- 
Kennedy  leg  involved  a  low  workload  scenario  with  routine  events.  The 
Kennedy  leg  involved  a  high  workload  scenario,  during  which  significant  data 
were  collected  about  aircrew  performance.  Some  of  the  areas  in  which  air¬ 
crew  performance  problems  were  observed  in  response  to  the  complications 
introduced  during  the  high  workload  scenario  included:  (a)  communications, 
(b)  decision-making,  (c)  resource  management,  (d)  crew  interaction  and 
integration, (e)  manuals  and  charts,  (f)  emergency  procedures,  and  (g) 
warning  system  design. 

One  important  finding  from  the  study  was  that  the  crews  that  managed 
available  resources  were  more  effective  in  handling  the  scenario.  The  role 
of  the  captain  was  particularly  important  in  setting  priorities  and  making 
use  of  the  other  crew  members. 
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This  study  helped,  in  part,  to  define  the  resource  management  problem. 

It  also  helped  to  develop  a  research  technique  and  the  full  mission 
simulation  approach  to  training  aircrews.  Anecdotal  data  collected  during 
the  simulation  runs  suggested  that  the  crews  were  captured  by  the  simulation 
and  appeared  to  respond  to  pressures  and  events  in  a  natural  manner.  Also, 
the  errors  and  problems  observed  corresponded  nicely  to  those  reported 
from  studies  of  real  flights,  and  to  incident  and  accident  data.  The  full 
mission  simulation  technique  has  been  adopted  by  several  airlines  to  meet 
requirements  for  recurrent  training,  and  experiences  at  Northwest  Orient  in 
particular  suggest  that  it  is  an  effective  training  technique. 

Proposed  work  under  this  general  research  program  includes  the  development 
of  a  training  program,  as  yet  not  fully  defined,  to  improve  aircrew 
resource  management  skills.  A  complementary  aspect  of  the  training  program 
will  be  to  measure  transfer  of  training, using  full  mission  scenarios  as 
the  transfer  vehicle  in  which  resource  management  is  evaluated.  The 
economics  of  aircrew  training  require  that  significant  advantages  be 
demonstrated  for  such  experimental  training  approaches,  both  in  terms  of 
immediate  transfer  and  long-term  performance,  if  airlines  are  to  modify  their 
current  training  practices. 

A  second  area  of  research  being  carried  out  by  the  NASA-Ames  group  deals 
with  the  level  of  automation  in  current  aircraft  systems.  Automation  is 
increasing  in  the  cockpit;  at  the  same  time,  human  monitoring  and  supervising 
of  automated  systems  will  still  be  required  for  the  foreseeable  future. 

Automation  is  a  two-edged  sword.  It  leads  to  increased  reliability  and 
precision,  but  it  also  introduces  another  level  of  man-machine  system 
problems.  Some  of  the  major  issues  are  listed  below: 
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(1)  Erosion  of  competence  (or  skill  atrophy)  as  a  result  of  lack  of 
utilization  of  skills. 

(2)  Selection  of  an  appropriate  level  of  system  reliability  - 
Intermediate  levels  of  system  reliability  may  introduce  a  false 
sense  of  complacency,  a  problem  which  may  not  occur  at  high  or 
low  levels  of  reliability.  False  alarm  rates  and  missed  alarms 
relate  to  this  issue. 

(3)  Level  of  communication  -  How  much  and  by  what  means  should  the 
automatic  system  tell  the  pilot  what  it  is  doing?  Will  system 
failures  be  detected  more  readily  when  the  pilot  is  an  active 
controller  or  a  passive  monitor?  Which  mode  will  best  facilitate 
diagnosis  of  a  problem  and  selection  of  a  recovery  response? 

(4)  How  can  automated  systems  be  designed  so  as  to  maintain  their 
effectiveness  in  spite  of  individual  differences  in  crew  capa¬ 
bilities  and  operating  style? 

(5)  Which  characteristics  of  automated  systems  should  be  included 
to  make  the  systems  sufficiently  "transparent"  to  the  pilot  so 
that  malfunctions  can  readily  be  detected? 

(6)  What  countermeasures  can  be  used  to  alleviate  the  problems 
listed  or  implied  above? 

(7)  Development  of  boredom,  followed  by  complacency,  when  crew 
functions  are  automated. 

(8)  Impact  of  automation  on  job  satisfaction. 
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One  goal  of  this  research  program  is  to  provide  human  factor  guidelines 
for  automation  of  cockpit  functions.  The  general  guidelines  and  working 
hypotheses  might  take  the  following  form: 


(1)  Lack  of  practice  (six  months)  does  not  affect  tracking  skills 
unless  the  tracking  tasks  require  velocity  compensation  (phase 
lead,  anticipation). 

(2)  Knowledge  of  complex  procedures  degrades  significantly  in 
three  months. 

(3)  Low  levels  of  workload  produce  large  variability  in  failure 
detection  response  (Some  people  do  better  at  low  levels  of 
workload;  others  do  worse.). 

(4)  Automatic  systems  should  perform  their  task  "as  the  pilot"  for 
more  efficient  monitoring  and  takeover. 

Some  hypothetical  examples  of  guidelines  specifically  for  cockpit  functions 
might  include: 

(1)  Autopilots  should  have  a  capability  for  entering  and  maintaining 
holding  patterns. 

(2)  Pilots  should  exercise  all  flight  control  system  modes  at  least 
once  in  three  months. 

(3)  Pilot  flying  approach  should  have  flight  director  on,  pilot  not 
flying:  flight  director  off. 

(4)  Fail-passive  autopilots  should  have  easily  interpretable  control 
laws  (for  better  failure  detection)  and  wind  shear  detectors. 
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These  issues  and  related  guidelines  indicate  some  of  the  research  topics 
that  will  be  pursued  by  the  NASA-Ames  group  in  the  near  future. 
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CASE  STUDY  OF  ACCIDENT  ANALYSIS 
AND  REPORTING 

James  Danaher 

National  Transportation  Safety  Board 

About  3:00  p.m.  on  April  27,  1976,  American  Airlines  Flight  625  overran 
the  departure  end  of  runway  9  after  landing  at  the  Harry  S.  Truman  Airport, 
Charlotte  Amalie,  St.  Thomas,  Virgin  Islands.  The  aircraft  struck  the 
instrument  landing  system  localizer  antenna,  crashed  through  a  chain  link 
fence,  and  came  to  rest  against  a  building  located  about  1,040  feet  beyond 
the  departure  end  of  the  runway.  The  aircraft  was  destroyed.  Of  the 
88  persons  aboard  the  aircraft,  35  passengers  and  2  flight  attendants  were 
killed.  Thirty-eight  other  persons  received  injuries  which  ranged  from 
minor  to  serious.  One  person  on  the  ground  was  injured  seriously. 

The  National  Transportation  Safety  Board  determined  that  the  probable 
cause  of  the  accident  was  the  captain’s  actions  and  his  judgment  in 
initiating  a  go-around  maneuver  with  insufficient  runway  remaining  after 
a  long  touchdown.  The  long  touchdown  was  attributed  to  a  deviation  from 
prescribed  landing  techniques  and  an  encounter  with  an  adverse  wind 
condition,  common  at  the  airport. 

The  non-availability  of  information  about  the  aircraft's  go-around  performance 
capabilities  was  a  factor  in  the  captain's  abortive  attempt  to  go-around 
after  a  long  landing. 

Selected  findings  from  the  analysis  of  this  accident  indicate: 

(1)  The  captain  did  not  follow  the  company  procedures  in  landing  at 
St.  Thomas.  The  company's  intent  was  to  require  a  40°  flap 
landing  configuration  for  all  landings  at  St.  Thomas  whenever  the 
headwind  component  did  not  exceed  20  kn  and  no  gusty  wind  conditions 
were  present. 
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(2)  The  use  of  30°  flap  instead  of  40°  flap  increased  the  landing 
roll,  provided  lower  drag,  lessened  the  decelerative  capability 
of  the  aircraft,  and  made  the  aircraft  more  susceptible  to 
atmospheric  or  aerodynamic  factors  which  could  produce  a  float. 

(3)  The  float  probably  resulted  from  either  an  updraft  encounter, 
or,  from  an  increase  in  lift  resulting  from  the  rotation  of  the 
aircraft,  or  an  increase  in  airspeed  as  a  result  of  a  rapid  cnange 
of  headwind,  or  a  combination  of  any  two  or  all  of  these  factors. 

(4)  A  successful  go-around  was  possible  immediately  upon  the  onset 
of  the  float,  after  the  wind  dropped,  and  most  probably  after 
the  wings  were  leveled.  This  capability  became  more  and  more 
marginal  as  the  float  and  engine  spool-down  continued. 

(5)  The  aircraft  touched  down  about  2,500  to  3,000  feet  beyond  the 
runway  threshold.  Based  on  these  distances,  it  could  have  been 
stopped  within  the  confines  of  the  remaining  runway,  but  a 
safe  go-around  could  not  be  made. 

(6)  Although  the  captain  realized  the  remaining  runway  was  critical 
with  regard  to  stopping  the  aircraft,  he  did  not  know  that  the 
remaining  runway  was  even  more  critical  with  regard  to  the 
execution  of  a  go-around. 

(7)  With  adequate  training  as  to  the  aircraft's  performance  capability 
and  with  training  environment  exposure  to  similar  situations, 

the  captain  may  have  reacted  immediately  to  stop  the  aircraft 
instead  of  attempting  a  go-around. 


Four  decisions  made  by  the  captain  should  be  noted  in  the  context  of 
situational  emergency  training: 
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(!)  The  choice  of  a  30°  flap  setting,  which  was  in  contradiction 
with  company  policy. 

(2)  The  decision  to  land  the  aircraft  instead  of  initiating  a 
go-around  prior  to  touchdown. 

(3)  The  decision  to  initiate  a  go-around  after  touching  down. 

(4)  The  decision  to  reject  the  go-around  maneuver  and  to  try  to 
stop  the  aircraft,  but  without  employing  maximum  stopping 
capability. 

Some  factors  which  may  have  contributed  to  the  difficulty  of  decision 
making  in  this  case  should  be  mentioned.  First,  the  company  directive 
to  initiate  a  go-around  when  touchdown  is  appreciably  beyond  the  aim 
point  does  not  specify  what  "appreciably"  means.  Second,  the  turbulence, 
which  caused  a  wing  to  dip  and  the  aircraft  to  float, constituted  a 
distraction  which  may  be  interpreted  as  delaying  the  captain's  realization 
of  the  distance  he  had  travelled  down  the  runway.  A  third  factor, 
referred  to  already,  was  the  non-availability  of  information  about  the 
aircraft's  go-around  capability  once  the  engines  were  spooled  down.  A 
fourth  factor  is  that  of  limited  prior  training  experience  in  go-arounds 
when  available  runway  distance  is  a  consideration.  A  fifth  factor  is  the 
utility  structure  of  the  captain  in  relation  to  employing  maximum  stopping 
capability;  this  maneuver  is  both  alarming  and  uncomfortable  from  the 
passenger  standpoint  and  hence  is  not  a  preferred  maneuver.  A  sixth 
factor  is  the  typical  pilot's  unfamiliarity  with  employing  maximum  stopping 
capability.  A  seventh  factor  is  the  natural  preference  for  choosing  a 
go-around  as  the  first  response.  This  is  generally  the  approved  procedure, 
which  pilots  are  rarely  criticized  for  employing,  when  a  landing  may 
involve  difficulties. 

This  particular  accident  is  a  widely  remembered  one.  The  foregoing 
review  of  the  accident  has  some  obvious  implications  for  training,  for 
procedures  and  policy  revision,  and  for  taking  other  precaution^  to  avoid 


a  repetition  of  it.  Since  the  analysis,  the  runway  has  been  lengthened 
and  obstacles  at  the  end  of  it  removed.  American  Airlines  has  modified 
its  procedures  and  now  requires  that  the  pilot  make  a  go-around  decision 
no  later  than  at  the  runway  threshold.  Although  American  Airlines  no 
longer  flies  into  St.  Thomas,  Eastern  Airlines  does,  and  their  training 
department  has  programmed  the  St.  Thomas  approach  into  its  simulator  and 
requires  pilots  to  be  checked  out  on  it  and  to  be  periodically  retrained 
on  it. 

This  accident  is  an  excellent  example  of  performance  degredation,  of  the 
breakdown  of  otherwise  well  ingrained  skills  in  an  unusual  context. 

(The  article,  "Managing  Emergencies  in  Jet  Transport  Operations,"  by 
Gerard  Bruggink,  in  the  Summer  1978  issue  of  Pilot,  presents  many  more 
related  examples  of  emergency  situations.)  My  hypothesis  about  this 
particular  incident  is  that  this  seasoned  captain  experienced  the  unexpected, 
reached  down  then  into  his  bag  of  tricks  to  correct  the  situation,  and 
aggravated  it  instead  of  correcting  it.  He  then  realized  that  an  overrun 
was  inevitable  and  an  accident  was  possible.  This  realization  may  have 
triggereo  the  response  which  cancelled  out  the  deeply  ingrained,  appropriate 
response  )f  braking,  reversing  and  lowering  the  nose. 

A  final  point,  or  generalization,  which  can  be  drawn  from  this  particular 
critical  incident  is  that  airline  training  is  deficient  in  the  area  of 
situational  emergency  training.  Most  airline  captains  receive  limited 
touch-and-go  training  and  not  under  conditions  which  are  representative 
of  live  operations  with  such  unfavorable  factors  as  were  present  in  this 
case.  The  flight  simulator  affords  an  excellent  and  safe  means  of 
accomplishing  such  training  and  more  could  be  done  using  the  simulator  as 
a  situational  emergency  training  device. 


SUMMATION  OF  ISSUES 


John  Lyman 
UCLA 

Four  general  categories  of  issues  arose  during  the  meeting,  and  these 
included  a  large  number  of  sub-issues.  Each  general  category  is  discussed 
in  turn.  The  first  is  the  issue  of  the  degree  of  rote  training  versus  the 
degree  of  abstract  or  decision  training  that  is  desirable.  We  can't  avoid 
dealing  with  rote  learning  of  specific  procedures.  This  is  clearly 
important.  Learning  procedural  rules  is  also  important,  however,  for 
they  allow  more  tailored  responses  under  varying  time  pressures  and  unique 
situational  factors.  The  recognition  of  what  part  of  a  set  of  procedural 
rules  to  use  or  not  to  use  takes  us  back  to  the  situation  recognition 
concepts  discussed  by  Ward  Edwards  and  others  earlier.  An  issue  raised  is 
the  distinction  between  template  matching  and  feature  recognition,  a 
fundamental  difference  of  approach  in  pattern  recognition  theory. 

Decisions  require  rules  for  operation  on  present  data,  while  rote  performance 
requires  specific  memory  of  what  to  do  in  a  predetermined  scenario  or 
script.  Humans  are  probably  better  oriented  to  the  former,  assembling 
features  of  a  situation,  then  applying  procedural  rules.  In  a  sense,  then, 
the  decision  aspects  of  emergency  response  are  continuous  and  need  to 
be  addressed  on  an  integrated  basis  together  with  requirements  for  rote 
response.  The  training  strategy  implied  is  to  facilitate  learning  to 
analyze  alternatives  for  assembling  the  features  of  a  situation  into 
successful  decision  states. 

The  second  class  of  issues  has  to  do  with  time  management.  The  time 
management  problem  in  the  air,  in  which  an  emergency  is  embedded  in  a  set 
of  other  demands  on  the  aircrew's  time,  can  be  a  principal  determiner  of 
response  capability  and  response  adequacy.  Ground  training  can  be  used 
to  reduce  reaction  time,  but  the  realism  of  ground  training  is  questionable 
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in  view  of  the  variable  and  complex  time  management  problem  in  the  air. 
Simulator  training  crosses  the  bridge  to  some  extent  between  the  ideal 
and  real  practice  of  decision  making,  but  more  study  is  needed  to  better 
simulate  airborne  time  management  conditions  as  they  affect  decision  making, 
as  well  as  to  improve  decision  making  under  such  pressures.  Time  critical 
vs.  time  relaxed  priorities  in  relation  to  decision  process  effectiveness 
is  probably  an  especially  important  area  for  study. 

A  third  general  theme  throughout  the  discussions  and  presentations  of 
this  meeting  is  the  notion  of  causal  chains .  The  problem  of  analyzing 
aircraft  accidents  validly  in  terms  of  contributing  factors  is  well  known. 

We  think  in  terms  of  a  sequence  of  incidents  leading  to  a  climactic  event, 
and  this  sequence  or  chain  is  very  hard  to  document  reliably  after  the 
fact.  It  appears  that  the  only  "accidents"  which  can  be  adequately 
studied  are  those  that  occur  in  simulators.  Generally,  the  reporting  and 
coding  of  accident  data  still  only  tell  us  what  happened,  not  how  it 
happened.  Safety  research  and  emergency  training  program  development  are 
both  hampered  by  the  seeming  inaccessibility  of  data  describing  these 
causal  chains.  Flight  recorders  have  been  a  productive  step,  but  clearly 
more  is  needed,  perhaps  including  distributed  sensors  recording  detailed 
data  about  the  aircraft  itself  and  its  relation  to  external  and  internal 
environmental  variables,  including  the  activities  of  the  flight  crew. 

Video  monitoring  should  also  be  considered. 

The  last  group  of  issues  involves  man-machine  personal  and  technical  factors. 

The  importance  of  specific  technical  factors  of  individual  systems  and 
subsystems  is  generally  recognized  in  emergency  situations  and  safety.  The 
wide  variation  in  individual  capability  is  another  area  of  importance. 

As  John  Lauber  and  Renwick  Curry  pointed  out,  some  excellent  pilots  just 
aren't  good  captains.  Both  psychological  and  physiological  factors  could 
be  studied  to  identify  superior  individuals  in  relation  to  their  assignment. 


32 


4i 


.  4 


fRECEDINO  PAGE  BUMUNOT  FUMED 


CURRENT  RESEARCH  IN  EMERGENCY 
DECISION  TRAINING 


I 


fHBCEDIAD  PAG S  BLMXmNOT  TILLED 


ACCIDENT  ANALYSIS  METHODOLOGY 

Major  Duncan  Dieter ly 
AFHRL  Technology  Office 
NASA-Ames  Research  Center 

The  work  described  below  is  part  of  a  larger  effort  being  carried  out 
at  NASA-Ames  to  study  and  improve  resources  management  in  a  multiman 
aircrew  operation.  In  this  particular  segment  of  the  resources 
management  program,  the  emphasis  is  on  human  error  problems  as  opposed 
to  equipment  problems  or  environmental  factors.  The  scope  of  interest 
extends  beyond  the  immediate  emergency  situation  and  includes  those 
performance  situations  in  which  an  impending  danger  is  not  an  issue 
though  an  accident  can  be  precipitated  by  such  a  series  of  "harmless" 
errors. 

The  theoretical  approach  presented  is  useful  for  analyzing  accidents  but 
is  not  intended  to  be  limited  to  accident  analysis.  It  is  a  general 
theory  of  problem  solving  and  decision  making  that  fits  this  type  of 
situation.  Our  program  has  three  objectives,  all  of  which  are  important: 

(1)  To  develop  a  technique  to  analyze  accidents  in  terms  of  the 
underlying  dimensions  of  human  error. 

(2)  To  use  the  results  of  analyses  of  accidents  by  this  technique 
to  develop  training  programs  aimed  at  reducing  human  errors. 

(3)  To  operationally  reduce  human  errors  made  by  aircrews  and 
accidents . 

The  theoretical  approach  to  the  analysis  of  accidents  integrates  two 
areas  of  research  and  conceptualization,  namely  problem  solving  and 
decision  making.  These  concepts  are  considered  as  functions  of  one  process. 
A  prescriptive  clarification  process  model  has,  thus,  been  established  to 
integrate  these  two  conceptual  areas.  It  consists  of  five  phases: 
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(1)  Identify  the  situation. 

(2)  Determine  options. 

(3)  Establish  outcomes. 

(4)  Evaluate/analyze. 

(5)  Select  option  and  implement. 

An  information  manipulation  function  is  a  critical  aspect  of  the 
clarification  process  model  whose  phases  are  highly  dependent  on  in¬ 
formation  flow.  The  information  manipulation  function  consists  of 
five  sub-functions: 

(1)  Attaining  information. 

(2)  Screening  information. 

(3)  Standardizing  information. 

(4)  Classifying  information. 

(5)  Storing  information. 

A  function  control  mechanism  is  postulated  as  governing  these  sub¬ 
functions  in  a  fashion  similar  to  the  command/control  mechanism  in  a 
computer.  The  information  manipulation  function  is  a  crude  model  of 
the  cognitive  capability  of  the  individual  and  this  element  of  the 
clarification  process  model  addresses  the  possible  constraints  of 
individual  differences  and  experience  in  resolving  particular  decision/ 
problem  conditions. 

Some  of  the  ideas  underlying  this  general  approach  should  be  described. 
First,  accidents  are  the  culmination  of  a  sequence  of  events;  if  some 
of  the  events  or  decisions  in  the  sequence  had  been  handled  differently, 
the  accident  would  have  been  avoided.  Second,  a  major  decision/problem 
state  consists  of  a  set  of  related  decision/problem  condition  sequences 
which  are  not  necessarily  symmetrical  when  compared  with  each  other.  Third, 
a  hierarchical  model,  which  is  a  modification  of  the  classical  decision 
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tree,  serves  to  incorporate  many  aspects  of  reality  that  the  decision 
tree  does  not  and,  as  a  result,  depicts  the  decision/problem  state  in 
a  more  useful  fashion.  The  hierarchical  model  permits  display  of  these 
features  of  decision/problem  states: 

(1)  Differences  in  importance  or  criticality  of  the  various 
individual  decisions. 

(2)  Differences  in  the  timing  of  events  in  each  decision/problem 
condition  sequence  and  among  sequences. 

(3)  Differences  among  sequences  in  the  number  of  decisions. 

(4)  Association  together  of  all  sequences  whose  outcome  is  suc¬ 
cessful  decision/problem  resolution  and  similar  separate 
association  of  those  sequences  which  result  in  failure. 

The  clarification  process  model  affords  a  means  to  look  at  the  process 
followed  by  individuals  and  aircrews  when  faced  with  various  situations 
and  to  identify  process  errors  and  information  problems.  With  such 
knowledge,  strategies  can  be  developed  and  taught  for  resolving  specific 
decision/problem  conditions  and  states  successfully.  From  a  research 
standpoint,  the  approach  outlined  here  has  clear  value  for  the  development 
of  a  taxonomy  of  decision/problem  conditions  in  relation  to  which 
generalized,  rather  than  situation  specific,  decision/problem  resolution 
skills  might  be  taught. 
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REQUIREMENTS  ANALYSIS  FOR  DECISION  TRATNING 


Joseph  Saleh 
’Perceptvonios ,  'no. 

The  work  I  am  going  to  briefly  describe  was  carried  out  as  part  of  an 
effort  sponsored  by  the  Naval  Training  Equipment  Center  in  Orlando, 

Florida  and  is  reported  more  fully  in  a  February  1978  technical  report 
("Analysis  of  Requirements  and  Methodology  for  Decision  Training  in 
Operational  Systems,"  NAVTRAEQUIPCEN  77-C-0005-1 ) .  In  this  presenta¬ 
tion,  I  will  present  an  overview  of  our  basic  methodology  for  deriving 
training  requirements,  and  Tony  Leal  will  provide  an  example  of  our 
application  of  it  for  ASW  helicopter  pilot  training. 

There  are  three  steps  essential  to  the  derivation  of  requirements  for  a 
decision  training  program.  They  are:  (1)  identification  of  decision 
tasks,  (2)  classification  of  the  identified  tasks,  and  (3)  association 
of  instructional  methods,  media  and  content  with  particular  tasks. 

The  identifioation  of  decision  tasks  comes  from  an  analysis  of  training 
materials,  procedural  doctrine,  and  interviews  with  experts.  We  generally 
find  that  decision  tasks  are  hidden  or  embeddedwithin  more  straight¬ 
forward,  larger  procedures.  For  instance,  part  of  a  task  might  be 
stated  as:  "Identify  the  contact  as  a  natural  object,  friendly  ship, 
or  enemy  ship."  The  operator  is  told  what  should  be  accomplished,  but 
not  how  to  do  it.  He  is  left  to  make  the  decision  on  his  own.  One 
approach  that  has  served  us  well  in  initially  screening  such  tasks  is 
the  use  of  keyword  analysis.  There  is  enough  consistency  in  military 
publications  and  informal  doctrine  that  certain  words  such  as  "analyze" 
and  "determine"  frequently  flag  tasks  which  have  decision  components. 


Classification  of  identified  decision  tasks  represents  the  second  major 
phase  of  activity.  We  employ  a  number  of  dimensions  for  classification. 
Primary  among  them  is  a  determination  of  which  major  components  of 
decision  making  are  involved  in  the  task.  The  following  schema  represents 
our  breakdown: 

(1)  Problem  structuring. 

a.  Formulation  of  alternatives. 

b.  Establishment  of  outcomes. 

(2)  Alternative  selection. 

a.  Assessment  of  utilities. 

b.  Assessment  of  probabilities. 

c.  Application  of  decision  rule. 

d.  Selection  of  best  alternative. 

Decision  tasks  can  involve  just  problem  structuring  which  we  term  a  Type 
1  decision  problem.  Once  the  problem  is  well  structured,  the  choice  of 
alternative  is  given  or  is  straightforward.  Other  decision  tasks  may 
involve  alternative  selection  only;  that  is,  the  identification  of 
alternatives  and  their  outcomes  is  obvious.  These  are  referred  to  as 
Type  2  decisions.  Those  decision  problems  which  involve  both  problem 
structuring  and  alternative  selection  are  referred  to  as  Type  3. 

We  also  classify  decision  tasks  according  to  several  other  attributes, 
as  exemplified  by  the  following  questions: 

(1)  Are  the  alternative  outcomes  single  attribute  or  multi¬ 
attribute? 
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(2)  Is  the  task  performed  by  an  individual  or  by  a  group? 

(3)  Is  the  task  static  over  time  or  dynamic  with  respect  to 

the  timing  of  actions  and  results? 

(4)  Is  the  task  a  one-shot  activity  or  is  it  repetitive? 

(5)  Is  the  task  performed  under  certainty  or  risk? 

(6)  Is  the  task  concretely  defined  or  ambiguous? 

(7)  Does  the  task  involve  decision  making  or  just  decision 
execution? 

(8)  Is  the  task  performed  under  time  stress  or  not? 

(9)  What  is  the  distribution  of  task  outcomes  with  respect 
to  negative  results? 

(10)  What  is  the  appropriate  decision  rule? 

The  last  major  phase  is  generation  of  instruction  r.d  iel  inoc.  Instruc¬ 
tional  content,  as  it  relates  to  the  various  aspects  of  decision  making 
listed  above,  is  selected  for  the  instructional  designer  in  the  form 
of  a  content  overview  with  examples.  This  is  integrated  in  training 
development  with  the  specific  aspects  of  the  real  world  decision  task 
which  underlie  the  problem.  Instructional  media  and  methodology  are 
also  recommended  to  the  developer  based  on  an  analysis  of  the  cognitive 
requirements  for  the  particular  decision  task. 

The  instructional  guideline  generation  system  was  realized  in  computerized 
form  as  a  small  program  that  interrogates  a  training  specialist  and  then 
produces  a  set  of  design  guidelines.  This  experimental  demonstration 
not  only  rapidly  produces  a  printed  prescriptive  guide,  but  also  unburdens 
the  designer  of  the  need  to  perform  the  calculations  that  support  the 
generation  process.  A  more  complete  description  of  the  methodology  and 
of  the  supporting  work  is  contained  in  the  technical  report,  and  I  refer 
you  to  it  for  further  details. 
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DECISION  TRAINING  FOR  ASW 
HELICOPTER  PILOTS 

Antonio  Leal 
Pevofq  tix  nies,  Inc. 

Perceptronics  recently  concluded  a  one-year  research  and  development 
program  in  the  decision  training  area  which  had  the  following  specific 
objectives: 

(1)  To  review,  from  a  decision  analysis  standpoint,  the  tasks  which 
constitute  the  Navy  SH-2F  (LAMPS)  ASW  Pilot/ATO  and  Sensor 
Operator  job  functions  and  identify  those  which  were  decision 
tasks. 

(2)  To  select  a  small  set  of  decision  tasks  from  the  larger  set  of 
decision  tasks  identified. 

(3)  To  describe  in  detail  one  specific  decision  task. 

(4)  To  develop  a  method  for  classifying  derision  tasks  and  establishing 
training  techniques  for  such  tasks. 

This  work  was  sponsored  by  the  Naval  Training  Equipment  Center  at  Orlando, 
Florida.  The  study  was  carried  out  at  the  Naval  Air  Station,  North 
Island,  which  has  an  excellent  helicopter  pilot  training  program.  We 
drew  on  a  variety  of  data  sources  in  carrying  out  our  analyses.  Courseware, 
Inc.  has  produced  an  excellent  training  syllabus  and  about  80  slide- 
tape  presentations  all  of  which  were  reviewed.  In  addition.  Courseware 
made  available  instructional  objective  hierarchies  which  summarize  the 
pilot's  functions  and  serve  as  the  background  for  detailed  training. 

We  also  interviewed  instructors  and  student  pilots,  all  of  whom  were 
extremely  cooperative. 

Our  analysis  revealed  that  within  this  very  comprehensive  course  there 
was  a  large  number  of  decision  tasks  which  were  not  procedural ized. 

In  essence,  at  certain  points  in  the  training  the  student  was  required  to 
realize  that  a  judgement  was  called  for  when  executing  a  procedure. 
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Development  of  the  capability  for  such  judgements  was  not  addressed  in 
the  didactic  instruction;  rather,  it  is  anticipated  that  students  will 
develop  the  capability  to  make  these  decisions  through  experience  on 
training  flights  and  through  conversations  with  instructors  and  fellow 
pilots. 

We  settled  on  one  particular  situation,  failure  of  the  engine,  and 
carried  out  a  detailed  decision  analysis  of  the  alternative  actions 
and  their  outcomes.  A  very  extensive  decision  tree  was  prepared  with 
the  aid  of  LAMPS  pilots  which  showed  the  probabilities  of  events  and 
the  utilities  for  the  various  outcomes.  From  this  particular  analysis, 
we  branched  out  to  identify  some  ways  to  describe  the  various  character¬ 
istics  of  emergency  and  routine  decisions.  Four  dimensions  emerged 
which  seemed  important  in  classifying  decisions  which  are  taught  to 
LAMPS  pilots: 

(1)  Well  defined  vs.  ambiguous. 

(2)  Single  attribute  vs.  multi-attribute. 

(3)  Time  pressured  vs.  time-relaxed. 

(4)  Static  vs.  dynamic. 

The  single  attribute/multi-attribute  dimension  refers  to  whether  the 
decision  outcome  has  one  or  more  value  dimensions  which  need  to  be  con¬ 
sidered  when  determining  payoff.  Static/dynamic  refers  to  whether  the 
consequences  of  the  decision  are  constant  over  time  or  not.  The  other 
two  dimensions  are  straightforward. 

We  looked  also  at  the  various  decision  rules  that  pilots  might  employ  in 
making  various  types  of  decisions.  These  included  among  others,  maximizing 
subjective  expected  utility,  lexicography,  and  satisficing.  Pilot 
interviews  revealed  that  their  decisions  generally  followed  one  of  these 
well-known  decision  rules,  and  the  one  used  depended  on  the  situation, 
but  they  were  not  aware  of  the  formal  rules  nor  were  they  consciously 
trying  to  follow  a  particular  rule. 


The  remainder  of  our  work  involved  two  efforts.  First,  we  matched  each 
of  the  classes  of  decisions,  categorized  along  the  dimensions  described, 
with  the  appropriate  decision  rule.  Next,  we  looked  at  instructional 
methods  and  media  to  match  these  to  particular  classes  of  decision 
tasks.  The  result  was  an  instructional  guidelines  generation  system 
which  is  a  procedure  to  take  a  specific  decision  task  and  provide  guide¬ 
lines  for  the  instructional  developer  which  specify:  (1)  instructional 
contents,  (2)  instructional  method,  and  (3)  instructional  media. 

Our  general  conclusions  were  that  the  types  of  decision  tasks  that  we 
encounter  could  be  effectively  analyzed  by  decision  analysis  techniques 
and  that  formal  decision  rules  were  compatible  with  existing  procedures. 
Also,  a  formalized  instructional  guildeine  generation  system  was  realized 
which  we  feel  has  applicability  to  other  areas  in  addition  to  LAMPS 
operations . 
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SITUATIONAL  EMERGENCY  TRAINING 

Rosemarie  Hopf-Weichel 
Percep tronics ,  Inc. 

Our  overall  program  objective  is  to  establish  a  theoretical  basis  for  a 
systematic  approach  to  decision  training,  specifically  for  aircraft  emer¬ 
gency  situations.  The  work  in  progress  is  sponsored  by  the  Air  Force 
Office  of  Scientific  Research  and  relates  to  earlier  work  on  situational 
emergency  training  done  at  Williams  AFB.  There  are  some  parallels  of  this 
project  in  the  work  described  by  Antonio  Leal  with  LAMPS  helicopter  pilots, 
namely  identifying  decision  training  requirements  within  the  context  of 
certain  aircraft  emergencies,  analyzing  the  instructional  requirements, 
and  generating  the  basis  for  more  effective  training.  A  particularly 
important  output  of  our  current  effort  will  be  the  development  of  guidelines 
for  generating  scenarios  for  emergency  training.  Another  important  aspect 
of  this  work  will  be  to  develop  means  to  monitor  the  effectiveness  of 
training  to  improve  it  and  to  establish  its  validity. 

The  basic  responsibility  of  a  pilot  in  an  emergency  is  to  analyze  the 
situation  and  take  proper  action.  In  relation  to  this,  we  are  hoping  to 
apply  decision  making  principles  both  for  problem  structuring  and 
alternative  selection.  Our  methodology  follows  generally  that  which  was 
outlined  by  Joseph  Saleh.  We  expect  to  rely  on  principles  from  decision 
theory  and  decision  training  in  order  to  analyze  emergency  decision  tasks 
and  to  develop  a  set  of  requirements  for  training.  We  will  focus  on 
development  of  personal  decision  rules,  on  training  to  utilize  relevant 
information,  and  on  development  of  scenarios  that  are  systematic  and 
comprehensive  in  terms  of  training  requirements. 

Our  methodology  involves  developing  a  taxonomy  of  emergency  situations, 
analyzing  certain  situations  in  terms  of  decision  making  requirements. 
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and  designing  guidelines  for  scenario  generation  which  match  emergency 
situations  to  decision  making  principles  and  paradigms.  The  information 
sources  we  have  used  include  the  literature  on  decision  theory  and  decision 
training,  accident  reports,  pilot  and  expert  interviews,  flight  and  training 
manuals,  and  site  visits  to  a  number  of  bases. 

One  useful  exercise  we  carried  out,  based  mainly  on  accident  reports,  was 
to  establish  a  skeleton  decision  structure  for  a  classic  emergency  situation, 
namely  fire  and  explosion  in  the  air.  A  skeleton  emergency  situation  was 
developed  from  review  of  reports  of  a  number  of  related  accidents  and  used 
to  structure  interviews  with  several  pilots  in  order  to  correct  it  and 
validate  it  for  accuracy  and  completeness. 

We  also  developed  a  matrix  of  emergencies  by  flight  phase  with  the  help  of 
subject  matter  experts,  particularly  the  staff  at  ONR  in  Pasadena.  The 
matrix  helped  us  identify  particular  situations  for  more  detailed  study 
as  prototypes  of  critical  decision  training  problems.  It  helped  us 
identify  those  attributes  of  particular  situations  which  we  might  be 
interested  in,  such  as:  safety  criticality,  time  criticality,  and  current 
decision-making  effectiveness. 

One  goal  we  have  is  to  develop  some  general  principles  for  generation  of 
emergency  situation  scenarios  that  incorporate  both  the  complexity  of  the 
emergency  itself,  and  the  decision-making  aspects  of  the  situation. 

In  part,  this  could  involve  the  identification  of  relevant  situational 
factors,  by  flight  phase,  for  various  emergencies  and  the  specification  of 
variations  in  these  factors  which  impact  on  complexity  as  discussed  above. 

In  conclusion,  the  approach  to  scenario  generation  we  are  taking  is  syste¬ 
matic  with  respect  to  malfunctions  as  well  as  decision  training;  it  allows 
for  graduated  difficulty  of  training  problems,  identifies  items  to  be 
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included  in  future  decision  aids,  should  facilitate  evaluation  of  decision 
making  skills,  and  will  involve  the  user  in  an  ongoing  dynamic  process  in 
the  development  of  guidelines  and  scenarios. 
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FORMAL  VERSUS  SITUATIONAL  MODELS 
OF  SKILLED  PERFORMANCE 

Hubert  and  Stuart  Dreyfus 
University  of  California ,  Berkeley 

Our  goal  is  to  understand  how  one  can  achieve  highly  skilled  performance 
both  in  coping  with  a  wide  variety  of  routine  situations  and  with  the 
abnormal  situations  known  as  emergencies.  Levels  of  skill  can  be  roughly 
classified  into  three  categories:  novice,  proficient  and  master.  Simply 
stated,  our  question  is  whether  skilled  performance  at  any  of  these  three 
levels  can  be  made  explicit  and  formalized. 

The  most  naive  approach  to  this  problem  is  to  suppose  that  skilled  per¬ 
formance  can  be  captured  or  described  in  the  form  of  a  list  of  specific 
procedures  applied  to  explicit  context-free  features.  A  simple  example 
of  this  approach  could  come  from  chess  playing  where  a  beginner  learns 
to  trade  pieces  so  as  to  maximize  material  balance  in  terms  of  the  point 
values  traditionally  assigned  to  the  various  pieces.  This  approach  is 
obviously  too  simple.  No  one  would  seriously  maintain  that  more  than 
novice  performance  could  result  from  following  rules  which  operate  on 
features  that  even  a  beginner  can  recognize. 

A  more  sophisticated  approach  would  involve  the  use  of  maxims.  Maxims 
are  procedures  whose  use  requires  a  thorough  familiarity  with  the  domain 
to  which  they  apply.  For  example,  a  chess  maxim  is:  avoid  an  unbalanced 
pawn  structure.  There  are,  of  course,  no  rules  by  which  a  novice  could 
determine  whether  a  pawn  structure  was  unbalanced.  Our  contention  is  that 
while  proficient  performance  can  be  attained  in  this  way,  mastery  cannot 
be  reached. 

Evidence  suggests  that  the  analytic,  information  processing  approach  to 
skill  development  actually  works  against  development  of  mastery  level 
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skill.  Study  of  top  level  performance  shows  that  given  natural  talent 
and  a  great  deal  of  experience  and  practice,  a  fundamental  change  takes 
place  in  the  performer.  This  change  might  be  described  as  entering  the 
world  of  the  activity.  For  example,  expert  pilots  no  longer  feel  that 
they  are  flying  the  plane,  but  rather  that  they  are  flying.  Flying  has 
become  like  walking,  something  one  does  without  detaching  oneself  from 
it.  The  chess  player  becomes  absorbed  in  the  world  of  the  game.  The 
tennis  player  no  longer  has  to  strain  to  keep  his  eye  on  the  ball,  but 
becomes  fascinated  by  what  the  ball  is  doing.  One  might  argue  that  the 
master  has  unconsciously  developed  and  is  using  better  maxims,  but  the 
adaptability  of  expert  performance  suggests  that  no  such  rigid  rules  are 
being  employed. 

There  is  evidence  from  studies  on  non-analytic  concept  formation  that 
giving  people  maxims  and  trying  to  sharpen  their  analytic  mind  actually 
inhibits  the  transition  from  proficiency  to  mastery.  From  our  perspective, 
nothing  but  talent,  considerable  experience  and  a  long  learning  curve 
plateau  can  lead  to  mastery.  If  one  wants  to  manage  training  programs  so 
that  master  performance  results,  we  strongly  advise  trainers  and  training 
designers  to  avoid  maxims,  which  engage  the  analytical  mind,  and  suggest 
they  speak  to  trainees  directly  from  within  their  own  world  of  expertise. 
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PERSONAL  FACTORS  IN  EMERGENCY 
DECISION  MAKING 

Carl  Castore 

Purdue  University  and  AFHBL/FT ,  Williams  AFB 

In  recent  years,  I  carried  out  a  series  of  studies  together  with  Dr. 

Siegfried  Streufert  on  various  aspects  of  the  decision  making/decision 
implementation  process.  Although  the  studies  were  not  done  in  the 
context  of  flying,  the  results  may  have  some  implications  for  aircrew 
emergency  procedures.  Briefly,  our  studies  looked  at  the  effects  of 
several  variables,  including  information  load  and  the  relevance  of  infor¬ 
mation.  The  study  setting  was  the  Tactical  and  Negotiations  Game,  an 
experimental  simulation  described  in  the  literature.  Some  of  the  aspects 
of  decision  making  studied  included  information  search  activities, 
respondent  decision  making,  and  integrated  decision  making. 

Of  particular  interest  to  the  aircraft  emergency  situation  is  the  case  in 
which  objective  external  criteria  are  not  readily  available  to  the  decision 
maker  in  the  form  of  feedback  to  evaluate  the  decision  and  take  corrective 
actions.  The  basic  results  of  these  laboratory  studies  for  that  case 
show  that  the  ability  of  individuals  to  take  individual  bits  of  information 
and  coordinate  them  into  integrated  decisions  is  best  at  moderate  levels 
of  information.  When  the  decision  maker's  responses  must  be  written  out, 
seven  to  ten  pieces  of  information  can  be  handled  well  in  a  twenty  to 
thirty  minute  period.  If  the  requirement  to  record  actions  is  removed, 
the  time  period  may  shrink  by  about  one  half.  In  any  case,  our  results 
show  that  it  is  difficult  for  individuals  to  process  very  much  new 
information  effectively  in  a  relatively  short  time  period,  one  which  is 
much  longer  than  that  found  in  many  aircraft  emergencies.  At  high  levels  of 
information  load,  integrated  decision  making  is  replaced  by  respondent 
decision  making,  a  situation  in  which  individuals  simply  resoond  to  each 
issue  as  it  arises  without  coordinating  information  or  coordinating  decisions. 


The  effects  of  irrelevant  information  are  particularly  illuminating. 

At  low  levels,  irrelevant  information  was  mostly  ignored,  assuming 
moderate-high  levels  of  relevant  information;  however,  if  overall  informa¬ 
tion  was  low,  irrelevant  information  was  integrated  with  available 
relevant  information.  Moderate  levels  of  irrelevant  information  triggered 
a  great  deal  of  information  search  activity,  as  well  as  attempts  at 
integrating  irrelevant  information  with  task  requirements.  High  levels 
of  irrelevant  information  produced  high  levels  of  information  search, 
coupled  with  high  levels  of  respondent  decision  making.  It  seems  that  at 
high  levels  of  information  load,  individuals  have  difficulty  sorting  out 
relevant  and  irrelevant  information,  but  continue  to  seek  out  information 
without  being  able  to  integrate  it  effectively. 

How  does  this  all  relate  to  the  aircraft  emergency?  In  my  work  at  Williams, 

I  am  exploring  possible  relationships,  relying  in  part  on  pilot  interviews 
for  interpretive  data.  One  finding  is  that  pilots  engage  in  a  great 
deal  of  preplanning.  They  anticipate  problems  and  rehearse  responses 
both  on  the  ground  and  in  flight  when  the  situation  permits.  This  allows 
the  pilot  to  stay  ahead  of  the  aircraft,  to  avoid  the  situation  in  which 
he  must  simply  react  without  time  to  integrate  new  information.  In  this 
context,  Boldface  represents  a  way  to  aid  the  less  experienced  pilot  when 
he  is  in  a  situation  whose  information  processing  demands  exceed  his 
capability.  Boldface  gives  that  pilot  something  to  do  right  away.  With  a 
more  experienced  pilot,  one  who  is  able  to  handle  the  cognitive  preprocessing 
or  analysis  of  what  can  go  wrong  based  on  his  detection  of  some  early, 
subtle  cues  and  in  light  of  his  knowledge  of  the  aircraft’s  systems,  a 
more  sophisticated  approach  to  emergency  training  seems  indicated. 

Another  of  our  laboratory  findings  that  seems  to  apply  to  the  emergency 
setting  is  that  people  delay  in  making  decisions  when  there  is  no  clear-cut, 
objective  criterion  or  feedback.  They  may  engage  in  far  more  information 
search  than  is  actually  needed.  In  the  multi-person  crew,  this  could  lead 
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to  coordination  problems  among  individuals  who  disagree  about  what  the 
actual  situation  is.  The  impact  may  be  to  hamper  decision  implementation, 
that  is  to  hinder  the  coordination  of  the  steps  necessary  to  resolve  the 
problem. 

A  problem  that  is  most  intriguing  from  a  research  standpoint  is  that  of 
avoiding  the  integration  of  irrelevant  information  into  a  decision. 
Individuals  are  more  likely  to  integrate  irrelevant  information  into  a 
decision  process  when  there  are  low  levels  of  information  involved  than 
otherwise.  The  resultant  decision  may  be  technically  correct,  given  the 
information  used,  but  can  lead  to  an  undesired  outcome.  Similarly,  the 
cases  in  which  aircrews  seem  to  focus  on  one  particular  piece  of  information, 
such  as  the  fixation  on  the  landing  gear  light  that  occurred  in  the  Eastern 
Airlines  accident  in  the  Everglades,  demonstrate  the  effects  of  failure  to 
coordinate  information  and  decision  making.  Although  I  don't  have  any 
ready  answers  to  improve  aircraft  emergency  procedures,  I  feel  that  a 
better  understanding  of  the  problems  of  aircraft  emergencies  from  the  stand¬ 
point  of  the  aircrews  themselves  will  increase  the  applicability  of  the 
research  I  have  briefly  described  to  this  area. 
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DECISION  TRAINING  NEEDS  FROM  THE 
STANDPOINT  OF  AIRCRAFT  ACCIDENT 
REPORTING  AND  RESEARCH 

Richard  Davis 
USC  Safety  Center 

Accident  investigation  reports  in  the  Air  Force  are  the  result  of  a 
detailed,  exhaustive  process  carried  out  by  a  board  of  officers  who  are 
supported  in  their  investigation  by  a  variety  of  technical  experts.  While 
the  job  these  boards  do  is  a  very  commendable  one,  training  research  and 
development  personnel  should  understand  the  limitations  on  these  reports 
and  also  on  the  data  which  is  eventually  stored. 

One  limitation  is  in  the  area  of  identification  of  accident  causes.  The 
current  data  bank  of  abstracted  Air  Force  reports  includes  only  what  might 
be  described  as  primary  causes.  Second  level,  or  root,  causes  are 
normally  not  included.  At  best,  root  causes  are  available  only  as  the 
result  of  tedious  analysis  of  the  full  reports  themselves.  It  is  very 
difficult,  therefore,  to  develop  an  in-depth  definition  of  training 
requirements  from  a  study  of  computerized  accident  report  summaries.  A 
second  limitation  has  to  do  with  the  encoding  process.  Reports  are 
summarized  for  entry  into  the  computer  by  encoders  who  are  not  experienced 
aircrew  members.  The  potential  loss  in  translation  is  great.  A  third 
factor  which  limits  the  usefulness  of  accident  reports  is  the  tremendous 
amount  of  scrutiny  that  reports  receive  up  through  the  chain  of  command 
before  they  are  released.  Knowing  that  this  critical  analysis  will  take 
place,  accident  boards  are  somewhat  reluctant  to  include  information 
which  is  not  the  equivalent  of  possessing  legal  sufficiency.  These  are 
some  of  the  facts  of  life  which  research  and  ISD  personnel  should  be  aware 
of  when  relying  on  current  accident  reports  and  data  banks. 

We  should  not  overlook  the  fact  that  in  the  past  twenty-five  years  we 
have  made  great  strides  in  preventing  aircraft  mishaps.  In  the  Air  Force, 
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the  accident  rate  has  dropped  from  about  60  major  accidents  per  100,000  flying 
hours  to  about  3.  The  major  improvement  has  been  in  the  hardware  area.  The 
next  challenge  is  to  deal  with  the  human  factors  contribution  to  accidents. 
Although  some  feel  that  the  human  factors  problem  cannot  be  dealt  with 
any  more  effectively  than  at  present,  unless  we  face  this  challenge  we'll 
never  know  if  progress  in  this  area  is  possible  or  not. 
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DECISION  TRAINING  NEEDS  FROM  THE 
STANDPOINT  OF  SIMULATOR  RESEARCH 
AND  TRAINING  PROGRAMS 


Elizabeth  Martin 
AFHRL/FT ,  Williams  AFP 

The  role  of  the  aviator  as  an  active  information  processor,  systems 
manager,  and  decision  maker  is  being  addressed  in  several  research  programs 
at  the  Flying  Training  Division.  Although  none  of  these  efforts  deals 
directly  with  emergency  training  research,  there  are  several  ongoing 
programs  investigating  various  facets  of  cognitive  processes  under 
differing  task  loading  and  stress  situations.  The  findings  of  these 
research  programs  should  be  directly  applicable  to  decision  training  needs 
across  a  variety  of  piloting  tasks,  including  emergency  situations.  A 
brief  description  of  these  programs  is  included  below,  along  with  a  point 
of  contact  for  each  effort. 

Within  the  basic  research  program,  two  efforts  deal  with  the  relationship 
between  behavioral  and  physiological  indices  of  pilot  functioning.  One 
effort  deals  with  establishing  relationships  between  a  set  of  physiological 
indices  of  attention  and  task  load  with  various  behavioral  measures  known 
to  be  sensitive  to  task  difficulty.  The  objective  is  to  define  a  set  of 
measures  which  is  most  consistently  related  to  behavioral  responses  on 
information  processing  tasks  which  represent  a  realistic  approximation  to 
actual  flight.  (Point  of  Contact:  Capt.  George  Buckland.) 

Another  related  effort  is  investigating  the  relationship  between  stress  in 
the  aircraft  and  stress  in  the  simulator  and  how  simulator  training  may  be 
used  to  alleviate  some  of  the  negative  stress  related  performance  effects 
observed  during  flight.  (Point  of  Contact:  Gary  Reid.) 

In  the  exploratory  development  research  program,  two  efforts  are  ongoing 
which  relate  to  the  pilot's  ability  to  deal  with  problems  encountered  in 
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flight.  One  effort,  entitled  Dynamic  Problem  Solving,  attempts  to  define 
and  teach  to  the  student  pilot  those  processes  characteristic  of  an 
experienced  pilot  that  enable  him  to  make  rapid  and  accurate  decisions 
relevant  to  changing  flight  requirements.  Initial  research  concentrates 
on  navigational  problems  in  terminal  area  orientation.  Application  to 
(simulated)  emergency  situations  is  one  of  the  test  conditions  which  will 
be  used  to  evaluate  the  instructional  procedure.  (Point  of  Contact: 

Capt.  David  Pohlman. ) 

Another  line  of  research  deals  with  performance  measurement  of  higher- 
order  aviating  skills  (e.g.,  situation  awareness,  judgement,  aircrew 
coordination),  which  are  not  typically  addressed  by  conventional  stick-and- 
rudder  measurement  sets.  The  research  approach  relies  on  the  ability  to 
create  situations  in  a  simulator  which  have  been  shown  to  require  the 
relevant  skill.  Through  repeated  exposures  to  these  situations,  pilots 
could  be  ranked  on  a  continuum  of  that  skill.  The  validity  of  the 
approach  would  be  determined  by  correlation  with  flight  evaluations. 

(Point  of  Contact:  Elizabeth  Martin.) 

In  almost  any  discussion  with  pilots  regarding  emergency  procedures 
training  or  performance  measurement,  the  concepts  of  judgement,  position 
awareness,  and  cockpit  discipline  arise.  The  commonly  held  tenet  is  that 
these  processes  cannot  be  taught  or  measured  directly,  and  that  they 
develop  almost  exclusively  as  a  function  of  flight  experience.  Indeed,  it 
seems  to  be  the  case  that  the  lack  of  these  processes  is  far  easier  to 
measure  than  their  presence.  All  too  many  hazardous  and  accident-producing 
situations  have  been  ascribed  to  lack  of  judgement.  In  addition  to 
reported  cases,  informal  inquiry  suggests  that  most  pilots  have  such  exper¬ 
iences.  (Fortunately,  the  most  frequent  results  are  acute  stress,  lingering 
embarrassment,  and  lessons  learned.)  It  seems  apparent  that  flying  training 
programs  require  increased  emphasis  on  the  development  of  judgement.  This 
presentation  addresses  the  potential  of  simulation  to  train  judgement. 
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The  distress  of  many  pilots  and  policy  makers  over  the  increasing  pressure 
to  replace  aircraft  flight  time  with  simulator  training  is  due,  in  part, 
to  the  fear  that  simulator  experience  cannot  produce  equivalent  growth 
in  airmanship  skills. 

The  primary  research  problem  seems  to  be  the  validity  of  simulation  for 
training  and  assessing  the  airmanship  skills  of  a  pilot.  The  standard 
research  technique  of  transfer  of  training  experiments  is,  for  the  most  part, 
infeasible.  Establishing  a  correlational  relationship  between  simulator 
behaviors  and  checkride  evaluations  is  possible,  but  unlikely  to  be  a 
sufficiently  sensitive  technique. 

A  thorough  analysis  of  the  factors  contributing  to  the  lack  of  judgement 
(position  awareness,  aircrew  coordination,  and  the  like)  is  a  necessary 
first  step.  For  example,  an  informal  preliminary  analysis  suggests  that 
many  situations  are  not  the  result  of  faulty  training  or  lack  of  awareness, 
but  rather  of  a  deliberate  decision  derived  from  subjective  assessment  of 
alternative  contingencies. 

A  preliminary  determination  of  which  factors  can  be  adequately  simulated 
and  trained  via  a  simulated  approach  would  be  the  next  step.  The  range  of 
simulation  concepts  need  not  be  restricted  to  the  full  mission  simulator 
but  should  include  such  techniques  as  mental  rehearsal  and  observational 
learning. 

Development  and  evaluation  of  experimental  training  packages  would  serve  as 
a  trial  evaluation.  Transfer  of  training  of  rule  governed  behavior  to 
novel  situations  would  constitute  the  test  environment.  Selection  of  the 
most  appropriate  problems  for  inclusion  in  operational  programs  would  follow 
from  the  previous  step. 

My  opinion  is  that  such  a  research  program  will  demonstrate  that  simulation 
can  play  a  positive  role  in  both  the  training  and  assessment  of  airmanship 
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components,  but  that  some  operational  contingencies  will  need  to  be 
modified  in  order  to  make  a  successful  transition  to  effective  training. 
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DECISION  TRAINING  NEEDS  FROM  THE  STANDPOINT 
OF  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT 


Andy  Gibbons 
Courseware ,  Inn. 

Developers  can  profit  from  an  awareness  of  decision  training  needs  both 
during  training  and  during  training  development. 

During  Training  Development:  There  is  a  growing  recognition  that  instruc¬ 
tional  systems  development  (ISD)  includes  more  than  producing  audio-visual 
materials.  Such  concepts  as  job  task  analysis,  instructional  sequencing, 
system  and  strategy  design,  and  formative  evaluation  are  now  widely 
accepted  as  important  and  valuable  elements  of  the  development  process. 
Proper  employment  of  development  techniques  can  produce  training  systems 
which  are  both  efficient  and  effective.  To  enhance  the  effectiveness  of 
emergency  training, the  developer  can  pay  attention  to  the  following  issues 
during  development: 

(1)  The  sequence  of  instruction  should  be  emphasized--the  series 
of  experiences  encountered  by  the  student  progressing  toward 
mastery.  Situational  Emergency  Training  (SET)  can  be  an 
effective  training  procedure  when  used  properly,  but  it  is 
only  one  step  of  several  the  student  passes  through  in 
becoming  proficient  in  emergency  management.  Developers 
should  pay  careful  attention  to  the  student's  experiences 
before  and  after  SET  to  insure  that  all  integrate  into  a 
steady  building  progression  of  knowledge  and  motor  skills. 

In  doing  this,  the  developer  should  insure:  (1)  that  all 
prerequisite  items  of  knowledge  have  been  identified  and 
instructed  prior  to  SET,  (2)  that  especially  critical  or 
difficult  component  behaviors  are  trained  in  isolation  prior 
to  training  in  context  with  other  behaviors  so  that  learning 
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of  the  component  does  not  interfere  with  integration  of 
the  larger  behavior,  (3)  that  an  appropriate  family  of 
training  devices  is  provided  through  systematic  device 
design  procedures,  particularly,  adequate  low-cost  inter¬ 
mediate  level  devices  which  can  relieve  scheduling  pressures 
on  more  expensive,  more  heavily-used  devices. 

(2)  Developers  should  insure  that  a  continually-functioning 
pipeline  has  been  set  up  to  supply  updated  information 
on  malfunction  identification  and  emergency  procedures 
which  can  be  incorporated  in  revised  versions  of  the 
instruction.  It  is  an  al 1 -too-common  experience  that  the 
training  system  is  the  last  to  know  this  information  which 
could  prevent  much  wasted  instructor  effort,  much  confusion, 
and  some  safety  hazards. 

(3)  Developers  should  devise  instructional  systems  which  main¬ 
tain  themselves  in  such  a  way  that  the  decision-making 
they  do  is  not  lost  in  the  future.  To  do  this,  an  audit 
trail  of  decisions  made  must  be  created,  and  guidelines 

for  making  future  decisions  made  must  be  created,  and  guide¬ 
lines  for  making  future  decisions  must  be  set  up,  if 
possible  through  procedural ization  of  the  decision  process. 
Too  often,  a  well -running  instructional  system  is  rendered 
inoperative  or  obsolete  by  decisions  made  which  invalidate 
the  planning  and  decision-making  of  the  developer. 

During  Training:  Careful  management  of  training  system  operation  can  also 
increase  the  effectiveness  of  emergency  management  training.  The  higher 
the  level  of  proficiency  expected  of  course  graduates,  the  more  careful 
the  management  of  the  training  must  be.  Some  suggested  areas  of  improve¬ 
ment  might  be  the  following: 
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(1)  Instructor  training  should  emphasize  techniques  which  have  a 
demonstrated  effectiveness  and  emphasize  the  importance 

of  following  those  instructional  procedures  whenever  possible. 
Instructors  should  be  discouraged  from  making  arbitrary  or 
idiosyncratic  changes  to  the  training  program.  Instead,  they 
should  be  given  a  well-defined  set  of  techniques  (SET  is 
one  example)  that  will  supply  the  student  with  appropriate 
levels  of  coaching,  feedback,  and  demonstration.  The 
instructor  need  not  be  taught  to  be  a  servant  to  the  tech¬ 
nique,  but  adequate  training  should  be  given  to  the  instructor 
to  demonstrate  the  benefits  of  using  it. 

(2)  Sufficient  resources  must  be  devoted  to  training,  and  cuts 
in  the  resources  must  be  weighed  in  terms  of  potentially 
reduced  student  capabilities.  Cuts  to  training  system 
resources  can  come  in  different  forms:  (1)  reduction  of 
total  equipment  and  personnel  resources,  (2)  expansion  of 
the  student  group,  forcing  allocation  of  fewer  resource  hours 
to  individual  students,  (3)  requirement  for  higher  graduate 
performance  levels  with  no  increase  in  resources.  All  of 
these  have  the  effect  of  moving  the  student  more  rapidly 
through  training,  forcing  the  student  to  take  larger  and 
larger  steps,  cutting  review  and  refresher  practice,  allowing 
less  time  for  the  consolidation  of  newly-learned  information. 
When  training  resources  are  reduced,  an  expected  reduction 

in  graduate  capabilities  should  also  be  expected  unless 
demonstrably  more  efficient  techniques  of  instruction  are 
employed  to  make  up  for  the  loss. 

(3)  The  training  system  must  be  insulated  from  external  pressures 
to  change  the  sequence  or  amount  of  training  without  full 
consideration  of  the  consequences.  Too  often,  changes  to 
the  training  are  mandated  without  careful  identification 
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of  need,  and  without  determination  of  the  implications  for 
graduate  capability.  The  changes  may  be  absolute  neces¬ 
sities,  but  they  should  be  accompanied  by  an  adjustment  in 
expectations  of  course  graduates. 
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DECISION  TRAINING  NEEDS  FROM  THE  STANDPOINT  OF 
PROCEDURAL  DOCTRINE  AND  THE  PRECREATION  OF 
EMERGENCY  SCENARIOS 

Stan  Roseoe 
University  of  Illinois 

There  are  three  platitudes  relevant  to  this  topic: 

(1)  To  train  pilots  to  make  good  emergency  decisions,  we  must 
first  know  how  systems  actually  work  and  fail.  This  includes 
knowing  how  pilots  fail  in  emergencies. 

(2)  We  must  learn  all  we  can  from  the  relatively  few  accidents 
that  do  occur,  not  only  what  happened,  but  why  it  happened. 

(3)  We  must  avoid  the  natural  tendency  to  narrow  the  blame  for 
accidents,  to  minimize  the  appearance  of  widespread  negligence. 

Corresponding  to  these  three  platitudes  are  three  needs: 

(1)  We  must  establish  the  optimum  procedural  doctrine  to  cope 
with  both  equipment  and  human  defections. 

(2)  We  must  establish  mission  training  scenarios  that  are 
pregnant  with  opportunities  for  bad  decisions  to  occur  and 
for  breakdowns  in  the  trained  procedures  that  we've  learned. 

(3)  We  need  to  extend  training  to  situation  recognition  that 
includes  recognition  of  your  own  temporary  impairment. 

In  the  St.  Thomas  incident,  the  procedural  doctrine  was  at  fault.  The 
directive  to  go-around  if  touchdown  were  to  be  beyound  1,000  feet  from 
threshold  could  not  be  carried  out  if  the  engines  were  spooled  down. 
Procedural  doctrine  should  be  tested  by  the  airlines  and/or  the  FAA  to 
determine  its  validity  before  it  is  issued. 


The  interaction  between  crew  members  and  the  rapid  pressurization  of 
the  cabin  prior  to  touchdown  in  the  St.  Thomas  incident  represent  factors 
that  may  have  contributed  directly  to  the  incident.  These  were  not 
fully  investigated  by  the  NTSB,  and  the  crew  members  themselves  did  not 
recall  them  as  significant.  Nevertheless,  we  must  create  an  investigative 
atmosphere  in  which  surviving  crews  are  encouraged  to  remember  everything 
that  they  can.  Perhaps  some  sort  of  protection  against  jeopardy  is 
needed  for  testimony  of  crew  members.  Some  form  of  encouragement  for 
complete  disclosure  is  needed  if  we  are  to  learn  all  we  can  through 
accident  analysis. 

This  incident  is  a  reminder  that  there  are  many  marginal  flying  situations 
around  the  world.  Those  situations  need  specific  situational  training 
scenarios  that  pre-create  emergency  conditions  as  widely  as  we  can  predict 
their  possible  occurrence.  At  the  same  time  procedural  doctrines,  such 
as  those  concerning  flap  settings  for  approach  and  go-around  at  St  Thomas, 
should  be  tested  for  such  marginal  situations  to  ensure  that  the  doctrine 
is  appropriate  to  specific  operational  situations. 


REPORTS  OF  DISCUSSION  GROUPS 
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DECISION  THEORETIC  APPLICATIONS  TO  EMERGENCY  SITUATION  ANALYSIS: 

REPORT  OF  DISCUSSION  GROUP  A 

Martin  Tolcott 
Office  of  Naval  Research 

This  workshop  group  focused  on  the  potential  that  decision  theory  holds 
for  the  training  of  aircrews.  Operational  training  personnel  in  this 
group  characterized  current  training  systems  as  follows: 

(1)  Current  training  now  results  in  adequate  levels  of  emergency 
decision  performance.  (Proposed  modifications  to  current 
practices  must  be  justified  as  improvements  with  measurable 
benefits  if  they  are  to  be  adopted.) 

(2)  Aircrew  emergency  functions  cover  the  continuum  from  routine 
responding  to  complex  decision  making. 

(3)  Training  techniques  must  be  diverse  to  allow  for  the  wide 
range  of  functions  required  of  aircrews  in  emergencies  (as 
noted  above). 

(4)  Theoretical  decision  concepts  must  be  linked  to  practical 
applications  in  order  to  gain  acceptance  and  use. 

(5)  Training  at  all  levels  of  prof iciency--novice,  journeyman, 
and  expert--is  important. 

Several  proposals  were  put  forth  by  the  workshop  group.  Those  achieving 
some  degree  of  consensus  are  listed  below: 

(1)  Pilots  must  make  inferences,  but  responses  should  be  as 
preplanned  as  possible  to  permit  rapid  (post-inferential) 
responses.  Training  in  making  inferences  could  usefully 
include  learning  to  make  probability  estimates,  provided 
that  this  is  done  for  specific  situations  (e.g.,  estimation 
of  likelihood  of  various  equipment  malfunctions).  Another 
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area  of  training  in  making  inferences  is  learning  to  update 
probabilities  as  new  data  becomes  available,  especially  the 
significance  of  data  dependencies  and  independencies  for 
situation  diagnosis.  Training  should  include  the  learning 
of  system  models  and  malfunctions  at  the  level  of  detail 
needed  for  malfunction  diagnosis,  and  not  at  a  more  detailed 
level.  The  statistical  processes  underlying  inference 
should  be  taught;  there  is  evidence  that  when  physicians  are 
trained  in  concepts  of  probability  and  utility  their  diag¬ 
nostic  capability  is  improved. 

(2)  Training  in  option  generation  should  be  considered  as  one 
way  to  avoid  the  natural  tendency  to  adhere  to  the  first 
option  that  comes  to  mind,  and  to  help  counteract  the  common 
finding  that  as  an  emergency  situation  proceeds,  the  option 
list  held  in  mind  tends  to  get  narrower  and  narrower. 

(3)  Analyses  of  operational  emergency  procedures  (e.g..  Boldface) 
show  that  such  procedures  do  include  decision  points,  even 
though  they  are  not  identified  as  such.  Decision  points 
should  be  clearly  identified  in  training;  also  two  types  of 
decisions  should  be  added  to  traditional  training:  (a)  the 
decision  that  an  emergency  does  exist  and  (b)  the  decision 
about  how  much  time  exists  to  resolve  the  emergency. 

(4)  Utilities  need  to  be  considered  somehow  in  emergency  training, 
although  utilities  are  hard  to  teach  and  not  very  useful 

when  an  immediate  response  is  required.  The  concepts  of 
priorities  for  action  and  trade-offs  are  considered  now  in 
operational  training.  An  extension  of  these  training  concepts 
to  include  utilities  and  risk-taking  might  be  helpful,  partic¬ 
ularly  in  responding  to  particular  scenarios  and  in  the  post¬ 
performance  review  by  instructor  and  student. 
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(5)  Decision  aiding  should  be  further  examined  for  its  relevance 
both  to  training  and  to  regular  mission  support.  Aiding 
can  take  a  number  of  forms,  including  display  of  symptoms, 
recognition  of  symptom  patterns,  and  display  of  appropriate 
response  sequences. 


A  few  issues  were  briefly  mentioned  that  deserve  further  research  attention 


(1)  Determination  of  the  most  difficult  tasks  or  decisions  to 
teach,  analysis  of  the  learning  problems  underlying  their 
difficulty,  and  development  of  instructional  strategies  for 
teaching  these  difficult-to-learn  tasks. 

(2)  Training  to  avoid  emergencies,  that  is,  training  to  recognize 
when  the  limits  of  the  aircraft  performance  envelope  are 
being  approached. 

(3)  Use  of  decision  theory  models  and  decision  trees  as  means  of 
communicating  decision  concepts  during  training  of  operational 
personnel . 
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ACCIDENT  ANALYSIS  AND  SAFETY  RESEARCH: 

REPORT  OF  DISCUSSION  GROUP  B 

Anchard  Zeller 

Hq  AF  Inspection  and  Safety  Center/SEL,  Norton  AFB 

An  old  saying  in  aviation  is  that  "Pilots  love  to  fly,  but  pilots  hate 
to  die."  This  saying  underscores  the  need  for  academicians  and  members 
of  the  commerical  research  community  to  ensure  that  their  efforts  are 
seen  as  relevant  to  operational  problems.  Many  of  the  aviation  research 
studies  done  each  year  fail  to  have  an  impact  on  operational  problems 
because  the  developers  and  researchers  do  not  come  up  with  products  that 
convince  operational  personnel  that  safety  is  enhanced.  A  cWer  dialog 
is  needed  between  the  research  and  the  operational  communities  to  bridge 
this  gap  and  ensure  that  useful  products  result,  which  operational  person¬ 
nel  will  adopt. 

Accidents  need  to  be  considered  in  a  systems  context.  The  machine,  the 
man,  and  the  interface  of  these  two  system  components  must  all  be  addressed. 
The  most  obvious  way  to  reduce  aircraft  emergency  decisions  is  to  improve 
the  reliability  of  the  aircraft.  A  second  approach  to  improve  safety  is 
to  improve  the  interface  of  man  and  aircraft  by  providing  good,  easy-to- 
interpret  information  and  responsive  controls. 

At  the  human  component  level  of  the  man-machine  system,  three  elements 
should  be  noted:  perception,  decision,  and  response.  Some  individuals 
seem  to  be  naturally  better  decision  makers;  others  have  biologically 
superior  perceptual  mechanisms.  The  implications  for  selection  are  clear. 
Those  individuals  who  are  better  able  to  concentrate  on  the  critical 
aspects  of  decision  making  should  be  sought  out  in  selection  processes. 


Two  other  related  factors  briefly  touched  on  in  this  group  were  emotions 
and  motivation.  The  first  is  particularly  important  and  deserves  further 
research.  Aircraft  emergencies  all  involve  some  level  of  stress;  the 
correspondence  between  training  decision  making  in  the  more  relaxed 
(simulated)  situation  and  performing  in  the  real  emergency  situation 
needs  to  be  studied  further  to  enhance  transfer  of  training. 


OPERATIONAL  TRAINING  PROGRAMS:  REPORT  OF 
DISCUSSION  GROUP  C 

Gary  Klein 
Klein  Associates 

This  workshop  group  covered  six  main  areas.  The  first  was  whether  formal 
decision  analysis  procedures  apply  to  aircraft  emergency  decision  train¬ 
ing  or  not.  The  consensus  was  that  no  matter  what  labels  were  chosen 
(decision  making,  problem  solving,  and  the  like),  they  did  not. 

A  second  topic  was  how  to  set  up  an  operational  emergency  decision  train¬ 
ing  program.  A  basic  starting  point  was  identified  as  establishing  the 
program's  desired  outcomes,  then  working  backwards  to  develop  the  training 
regimen.  Evaluations  of  such  programs  should  be  of  the  control  group  vs. 
experimental  group  type,  rather  than  using  the  transfer  of  training  approach. 

Performance  in  training  should  be  measured  in  terms  of  reactions  to 
scenarios.  This  raises  the  question  of  scenario  selection;  operational 
training  personnel  felt  it  would  not  be  difficult  to  select  appropriate 
scenarios,  while  others  recommended  the  selection  be  based  on  analysis  of 
accident  data  to  identify  the  important  training  areas. 

A  fourth  area  of  discussion  was  the  problem  of  defining  an  emergency.  It 
was  suggested  that  anything  which  had  potential  for  an  emergency  be 
included  in  a  broad  classification,  but  there  are  practical  problems  with 
this  idea.  Little  consensus  emerged  with  respect  to  a  satisfactory 
definition.  All  agreed  that  we  lack  a  generally  acceptable  definition. 

The  concept  of  emergency  interacts  with  many  conceptual  variables  such 
as  pilot  skill,  preceding  and  concurrent  events,  and  the  like. 

The  fifth  area  of  discussion  covered  what  the  ideal  emergency  decision 
training  program  content  might  include.  Abstract  training  on  decision 
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analysis  was  ruled  out,  as  noted  earlier.  A  straightforward  approach 
was  recommended  in  which  trainees  would  be  trained  on  the  same  specific 
emergency  situations  that  they  were  to  be  evaluated  on.  Thus,  the 
standard  approach  to  skill  development  used  in  any  other  training  program 
is  recommended. 

A  final  arp-  ■•ealt  with  by  the  group  was  training  considerations  specific 
to  aircraft  e*  ..vgencies.  They  are  listed  below: 

(1)  While  it  may  be  possible  to  develop  psychological  stress  in 
simulators,  it  currently  isn't  possible  to  introduce  physio¬ 
logical  stress  (e.g.,  high  G  forces,  nystagmus)  in  simulators. 
Perhaps  one  way  to  prepare  aircrews  for  physiological 
stresses  during  aircraft  emergencies  would  be  to  study  their 
effects  on  decision  processes  and  at  least  acquaint  crew 
members  with  the  possible  interactions  that  may  occur. 

(2)  Task  analyses  of  emergencies  should  include  identification 
of  the  special  cues  (sound,  motion)  that  need  to  be  included 
in  training  programs.  Such  cues  are  typically  not  available 
in  manufacturers'  information  or  elsewhere. 

(3)  Collection  of  aircraft  performance  data  early  in  the  manu¬ 
facturing  process  would  facilitate  the  training  development 
process.  Such  information  should  include  assessment  of 
interactions  between  components  made  by  different  subcon¬ 
tractors. 

(4)  A  question  yet  to  be  answered  in  training  development  is: 

"What  specific  knowledge  does  a  pilot  need  to  analyze  complex 
situations  and  predict  the  outcomes  of  different  reactions?" 

A  related  question  is:  "How  do  ISD  assumptions  affect  this 
knowledge  base?"  The  typical  ISD  decision  to  eliminate  all 
academic  ("nice  to  know")  information  may  undermine  the 
development  of  the  pilot's  basis  for  complex  analysis. 
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(5)  In  the  context  of  time  management  and  prioritization,  the 
group  discussed  the  need  to  blend  two  tasks:  (a)  flying 
the  plane  and  (b)  responding  to  the  emergency.  Training 
emergency  procedures  out  of  context,  such  as  in  the  CPT, 
seems  to  ensure  that  pilots  will  respond  to  emergencies  in 
a  focused  way  and  forget  the  larger  picture. 

(6)  The  question  of  looking  at  other  devices  besides  simulators 
for  emergency  decision  training  was  raised,  but  no  specific 
alternatives  were  identified. 

(7)  A  final  conclusion  was  that  management  decisions  should  be 
scrutinized  to  find  ways: 

(a)  To  make  instructor  pilots  available  for  training 
rather  than  for  other  activities. 

(b)  To  make  training  more  efficient. 

(c)  To  interface  emergency  training  with  overall  mission 
training. 
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PRECEDING  PAOE  BLANK- MOT  711ME D 


THIRTY  MINUTES  OVER  FLORIDA 


Rosemarie  Hopf-Weichel 
Perceptronics ,  Inc. 


INTRODUCTION 


The  following  incident  was  reported  by  Lieutenant  Mike  Bryant,  USN,  one 
of  the  attendees  at  the  Aircrew  Emergency  Decision  Training  Conference. 
The  events  reported  occurred  on  a  routine  S-3  training  flight  that  was 
conducted  just  ten  days  after  the  conference.  The  incident  not  only 
may  hold  some  human  interest  value  for  those  who  heard  Lieutenant  Bryant 
describe  his  views  on  emergency  training  at  the  conference,  but  may 
sc  we  to  i  i  lusircLc  the  type  of  situations  and  related  decisions  which 
emergency  decision  training  programs  can  address. 

INCIDENT 

On  December  9,  1978,  Lieutenant  Bryant  was  conducting  a  routine  training 
flight  in  an  S-3  over  Florida.  He  is  an  experienced  S-3  pilot,  having 
logged  some  2,000  flying  hours  in  S-3s  and  havina  experienced  eleven 
engine  shutdowns  while  in  flight.  He  had  a  student  as  a  co-pilot,  was 
45  minutes  into  the  flight,  had  leveled  off  at  28", 000  feet,  and  was  at 
normal  cruise  speed  (about  0.65  Mach)  when  he  experienced  a  pending  dual 
engit.->  failure. 

Lieutenant  Bryant  and  his  co-pilot  were  preparing  to  put  on  their  oxygen 
masks,  a  precautionary  procedure  required  above  27,000  feet,  when  they 
began  to  smell  smoke  similar  to  exhaust  gas  in-  the  aircraft.  The  smell 
of  smoke  continued  for  a  few  minutes,  then  the  number  two  engine  gauges 
started  to  fluctuate  a  bit.  The  probability  of  getting  a  whiff  of 
Environmental  Control  System  (ECS)  exhaust  in  the  cockpit  is  quite  high; 
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it  happens  frequently  in  the  S-3,  but  usually  at  lower  altitudes.  It  is 
also  common  for  a  gauge  to  fluctuate  a  little,  as  the  result  of  a  loose 
wire,  for  example;  but  it  is  not  common  for  all  or  most  of  the  gauges  to 
fluctuate  at  the  same  time. 

Although  Lieutenant  Bryant  was  not  alarmed,  he  was  aware  that  something 
was  probably  amiss.  There  was  a  little  more  smoke  and  fluctuation  of 
the  gauges,  and  suddenly  the  smoke  increased  greatly  and  all  the  gauges, 
except  the  oil  pressure  gauge,  were  fluctuating  wildly.  These  events 
strongly  suggested  that  engine  number  two  was  coming  apart.  Lieutenant 
Bryant  had  at  this  point  two  options  --  to  shut  down  engine  number  two 
or  to  continue  to  operate  with  both  engines.  The  first  course  would 
reduce  the  development  of  further  damage  or  problems  with  engine  number 
two  and  involved  little  risk  since  the  S-3  flies  well  with  only  one  engine, 
assuming  there  are  no  other  serious  malfunctions.  The  latter  option  runs 
the  risk  of  fire,  explosion,  or  flame-out.  Lieutenant  Bryant  quickly 
chose  the  first  option.  He  shut  down  engine  number  two  and  started  his 
descent  to  avoid  pressurization  problems. 

After  shutting  down  an  engine,  standard  (Boldface)  procedure  requires 
the  pilot  to  land  as  soon  as  possible,  unless  there  are  extenuating 
circumstances.  In  this  case,  there  were  no  contraindications  to  landing 
and  his  next  decision  was  to  evaluate  the  various  landing  possibilities 
and  select  one.  There  are  several  factors  to  consider  in  choosing  an 
emergency  landing  site.  The  most  important  of  these  is  distance,  in 
compliance  with  the  requirement  to  land  as  soon  as  possible,  although 
distance  must  be  weighted  against  weather  conditions,  availability  of 
arresting  gear  and  crash  crews,  the  type  of  emergency,  and  similar 
considerations. 
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For  Lieutenant  Bryant,  the  choice  was  fairly  simple.  There  were  two 
Navy  bases  and  two  Air  Force  bases  in  his  general  vicinity.  All  things 
being  equal,  he  would  have  selected  a  Navy  base  since  it  would  have 
appropriate  maintenance  facilities.  Lieutenant  Bryant's  maintenance 
crew  was  at  Cecil,  150  miles  away,  and  was  his  preferred  choice  with 
respect  to  available  facilities.  There  was,  however,  a  cold  front 
between  his  position  and  Cecil,  and  it  was  raining  at  Cecil.  The  other 
Navy  base,  Pensacola,  was  100  miles  away.  Both  of  these  fields  were 
further  from  him  than  the  two  Air  Force  bases:  Eglin,  about  80  miles 
away,  and  Tyndall,  about  50  miles  away.  In  this  case  Tyndall  was  the 
best  option  with  respect  to  all  remaining  factors.  If  Tyndall  had  been 
closed  because  of  weather,  the  second  choice  would  have  been  Pensacola. 
Eglin  is  undesirable  for  an  emergency  landing  because  it  has  several 
landing  areas  and  a  pilot  cannot  kraw  ahead  of  time  where  he  will  be 
directed  to  land.  "They  can  vector  you  to  maybe  90  miles  away,"  accord¬ 
ing  to  Lieutenant  Bryant. 

Arresting  gear  was  available  at  all  four  airfields.  For  this  case,  it 
represented  an  extra  safeguard.  If  engine  number  one  had  failed  instead 
of  engine  number  two,  arresting  gear  would  have  been  mandatory.  The 
functioning  of  the  utility  hydraulics,  which  include  the  main  brake  system, 
nosewheel  steering,  and  landing  gear  extension,  depends  on  engine  number 
one. 

Following  shut-down  of  engine  number  two.  Lieutenant  Bryant  had  declared 
an  emergency  with  Jacksonville  Center,  the  controlling  agency  that  provided 
him  with  distances  to  the  nearby  airfields.  After  considering  the  factors 
described  above,  he  decided  to  land  at  Tyndall  and  remained  in  radio 
contact  with  the  center  as  he  proceeded  toward  it.  At  about  9,000  feet 
and  some  20  miles  from  Tyndall,  the  situation  worsened.  The  entire  plane 
started  to  vibrate.  There  was  a  great  deal  of  noise,  and  engine  number  one 


appeared  to  be  failing.  These  cues  were  much  more  striking  than  the  cues 
preceding  shut-down  of  engine  number  two  and  are  typical  of  an  impending 
catastrophic  engine  failure.  The  best  course  of  action  appeared  to  be  to 
restart  engine  number  two  if  possible  and  then  shut  down  engine  number 
one.  Three  attempts  at  restart  of  engine  number  two  were  made,  but  with¬ 
out  success. 

Since  engine  number  two  could  not  be  restarted.  Lieutenant  Bryant  consid¬ 
ered  three  options:  attempting  to  land  immediately  on  the  nearest  highway, 
ejecting,  or  continuing  to  fly  towards  Tyndall.  He  decided  against 
attempting  to  land  immediately  or  ejecting.  His  strategy  was  to  continue 
flying  as  long  as  things  "got  rio  worse"  and  to  eject  if  they  did.  His 
criteria  for  ejection  were  subjective  in  that  they  depended  on  the  percep¬ 
tion  of  a  "significant"  change  in  noise,  vibration  or  gauge  readings  for 
engine  number  one. 

Lieutenant  Bryant  continued  toward  Tyndall  without  further  deteriorations 
of  the  situation.  As  he  neared  the  field  it  was  necessary  to  select  an 
approach  for  landing.  The  normal  procedure  would  be  a  10-mile,  straight- 
in,  radar-controlled  approach.  Conditions  permitted  a  visual,  3-mile, 
straight-in  approach  which  he  selected  to  expedite  landing. 

During  the  hectic  conditions  of  the  last  20  miles,  Lieutenant  Bryant  and 
his  co-pilot  managed  to  follow  all  the  checklist  procedures  for  a  precau¬ 
tionary  landing  except  for  dropping  gas.  They  did  so  on  final  approach 
and  landed  without  problems.  Immediately  after  landing,  the  cockpit 
filled  with  smoke.  Lieutenant  Bryant  shut  down  engine  number  one  and 
the  auxiliary  power  unit,  and  he  an  his  co-pilot  left  the  cockpit  head 
first,  fearing  fire.  After  verifying  that  the  engine  was  not  on  fire, 
Lieutenant  Bryant  returned  to  the  cockpit  and  the  airplane  was  towed  to 
a  hangar. 
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Lieutenant  Bryant  is  qualified  as  a  maintenance  check  pilot.  He  contacted 
his  maintenance  crew  at  Cecil  by  telephone  and  with  their  assistance 
discovered  that  a  failure  had  occurred  in  the  Environmental  Control 
System  turbine.  The  failure  had  caused  a  backup  in  the  bleed  air  system, 
producing  smoke  in  the  cockpit.  This  is  not  a  classic  turbine  failure;  in 
fact,  the  ECS  turbine  had  never  failed  in  this  particular  way  before. 

After  Lieutenant  Bryant  discovered  the  true  nature  of  the  malfunction, 
he  determined  that  the  aircraft  could  be  safely  flown  to  Cecil  as  long 
as  the  ECS  was  not  in  use,  and  returned  to  base  without  further  problems. 

COMMENT 

The  entire  incident  lasted  about  thirty  minutes.  The  failure  of  the  ECS 
turbine  produced  symptoms  characteristic  of  a  dual  engine  failure. 

Although  the  true  nature  of  the  S-3  malfunction  was  not  determined  until 
the  emergency  had  passed,  the  pilot  was  still  able  to  deal  effectively 
with  an  evolving  series  of  decisions  and  managed  the  overall  situation 
so  that  risks  to  life  and  aircraft  were  minimized. 

It  is  interesting  to  note  that  the  emergency  that  Lieutenant  Bryant  faced 
was  not  a  unitary  decision  problem  which  could  be  easily  solved  by  appli¬ 
cation  of  a  single,  standard  response  procedure.  Boldface  procedures 
were  important  in  the  management  of  the  emergency,  but  in  themselves 
were  not  sufficient  for  resolution  of  the  situation.  There  were,  in 
fact,  a  series  of  decisions  made  by  Lieutenant  Bryant,  some  of  which 
primarily  involved  diagnosis  and  others  which  mainly  involved  structuring 
action  alternatives  and  choosing  among  them.  In  addition  to  decision 
making,  the  pilot  was  required  to  execute  the  results  of  his  decision 
making.  All  of  these  phases  of  activity  (situation  diagnosis/problem 
structuring,  alternative  generation/selection,  and  decision  execution) 
should  be  addressed  in  emergency  decision  training  programs,  both  as 
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separate  functions  and  as  part  of  an  integrated  approach  to  the  manage¬ 
ment  of  an  emergency. 

The  role  of  experience  in  this  particular  emergency  should  not  be  over¬ 
looked.  The  pilot's  experience  led  him  to  the  reasonable  assumption  that 
engine  number  two  was  failing  and  the  decision  to  shut  it  down  was  then 
rapidly  made.  However,  his  experience  did  not  include  exposure  to  the 
rare  event  of  an  ECS  turbine  malfunctioning  in  this  particular  way.  As 
a  result,  he  did  not  entertain  other  hypotheses  besides  engine  failure  or 
feel  that  it  was  worth  the  risk  to  continue  operating  engine  number  two 
long  enough  to  obtain  disconfirming  evidence  (e.g.,  to  confirm  the  normal 
reading  of  the  oil  pressure  gauge  over  a  longer  period  of  observation). 

He  chose  to  minimize  maximum  loss  by  shutting  down  engine  number  two. 
Experience  was  also  a  factor  in  the  rapid  evaluation  of  landing  alternatives 
and  selection  of  the  best  site.  Finally,  experience  was  a  factor  in  estab¬ 
lishment  of  the  personal  decision  rule  to  conside'  ejection  only  if  the 
cues  from  engine  number  one  got  "significantly"  worse. 

Emergency  decision  training  programs  can  address  the  experience  factor  in 
various  ways.  One  is  to  adjust  the  complexity  of  training  problems  to 
match  the  sophistication  of  the  trainee  so  that  learning  is  optimal. 

Another  is  to  ensure  that  trainees  are  exposed  to  realistic  base  rates 
of  malfunctions  and  other  emergencies  and  to  various  combinations  of 
situational  factors.  In  this  way,  acquisition  of  skills  in  comprehensive 
decision  making  and  decision  execution  will  be  facilitated.  The  impor¬ 
tance  of  designing  training  scenarios  in  light  of  such  principles  is 
obvious. 

A  final  point  to  be  made  is  that  incidents  such  as  this  one  represent  a 
valuable  source  of  information  both  for  revising  emergency  procedures  and 
for  generating  training  problems  and  scenarios.  Following  his  experience, 
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Lieutenant  Bryant  gave  a  seminar  to  other  S-3  pilots  and  wrote  up  the 
incident  for  the  Navy  Unsatisfactory  Material  Reporting  System  to  ensure 
dissemination  of  the  experience.  He  also  revised  his  personal  decision 
rules,  given  a  recurrence  of  these  same  symptoms,  namely  to  include 
turning  off  the  ECS  in  order  to  eliminate  ECS  malfunction  as  one  competing 
hypothesis  to  engine  failure. 
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Panelists:  Current  and  Former  Pilots,  Instructor  Pilots  and 
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Percep tronics 
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Programs  -  Elizabeth  Martin,  AFHRL,  Williams  AFB 

C.  From  the  Standpoint  of  Instructional  Systems  Development  - 
Andy  Gibbons,  Courseware 

D.  From  the  Standpoint  of  Procedural  Doctrine  and  the  Precreation 
of  Emergency  Scenarios  -  Stan  Roscoe,  University  of  Illinois 

1050  Case  Study  of  Accident  Analysis  and  Reporting 

James  Danaher,  National  Transportation  Safety  Board 

1400  Small  Group  Meetings 

A.  Decision  Theoretic  Applications  to  Emergency  Situation  Analysis  - 
Leader:  Martin  Tolcott,  ONR 

B.  Accident  Analysis  and  Safety  Research  -  Leader:  Anchard 
Zeller,  Office  of  the  Inspector  General,  Norton  AFB 

C.  Operational  Training  Programs  -  Leader:  Gary  Klein,  Klein  Associates 

2000  Training  Technology  Demonstration 


THURSDAY,  NOVEMBER  30,  1978 

0900  Resource  Management  in  Present  and  Future  Aircraft  Operations 

John  Lauber  and  Renwick  Curry,  NASA-Ames  Research  Center 

1020  Reports  of  Discussion  Groups 

Moderator:  Gershon  Weltman,  Perceptronics 
Panelists:  Discussion  Group  Leaders 

1120  Summation  of  Issues 

John  Lyman,  UCLA 

1150  Concluding  Remarks 

Jack  Thorpe,  AFOSR 


1200  Adjourn 
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